[jira] [Created] (NIFI-2798) Upgrade Zookeeper version due to CVE-2016-5017

[Andy LoPresto (JIRA)]

Andy LoPresto created NIFI-2798:
---

 Summary: Upgrade Zookeeper version due to CVE-2016-5017
 Key: NIFI-2798
 URL: https://issues.apache.org/jira/browse/NIFI-2798
 Project: Apache NiFi
  Issue Type: Task
  Components: Core Framework
Affects Versions: 0.7.0, 1.0.0
Reporter: Andy LoPresto
Assignee: Mark Payne
Priority: Critical
 Fix For: 1.1.0, 0.8.0


The currently used version of Zookeeper {{3.4.6}} is subject to a buffer 
overflow attack using the C command-line interface (documented as CVE-2016-5017 
[1]). Version {{3.4.9}} patches this issue. In {{nifi/pom.xml}}, this version 
number should be updated, and basic compatibility/smoke tests should be run to 
ensure no new issues are introduced by the version increment. 

[1] https://zookeeper.apache.org/security.html#CVE-2016-5017



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi-minifi pull request #33: Minifi 108

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-minifi/pull/33


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-1841) SQLFeatureNotSupportedException with ExecuteSQL and Oracle Driver

[Bryan Bende (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Bende updated NIFI-1841:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> SQLFeatureNotSupportedException with ExecuteSQL and Oracle Driver
> -
>
> Key: NIFI-1841
> URL: https://issues.apache.org/jira/browse/NIFI-1841
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Bryan Bende
>Assignee: Matt Burgess
>Priority: Minor
> Fix For: 1.1.0
>
>
> Using ExecuteSQL with the Oracle ojdbc7.jar, the following exception is 
> produced:
> {code}
> java.sql.SQLFeatureNotSupportedException: Unsupported feature 
> at oracle.jdbc.driver.ClobAccessor.getBytes(ClobAccessor.java:534) ~[na:na] 
> at 
> oracle.jdbc.driver.GeneratedStatement.getBytes(GeneratedStatement.java:151) 
> ~[na:na] 
> at 
> oracle.jdbc.driver.GeneratedScrollableResultSet.getBytes(GeneratedScrollableResultSet.java:326)
>  ~[na:na] 
> at 
> org.apache.commons.dbcp.DelegatingResultSet.getBytes(DelegatingResultSet.java:241)
>  ~[na:na] 
> at 
> org.apache.commons.dbcp.DelegatingResultSet.getBytes(DelegatingResultSet.java:241)
>  ~[na:na] 
> at 
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:105)
>  ~[nifi-standard-processors-0.6.1.jar:0.6.1] 
> at 
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:72)
>  ~[nifi-standard-processors-0.6.1.jar:0.6.1] 
> at 
> org.apache.nifi.processors.standard.ExecuteSQL$1.process(ExecuteSQL.java:158) 
> ~[nifi-standard-processors-0.6.1.jar:0.6.1] 
> {code}
> The issue appears to be related to calling ResultSet getBytes().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (NIFI-2788) Cluster icon in the menu doesn't have a consistent size

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan reassigned NIFI-2788:
-

Assignee: Scott Aslan

> Cluster icon in the menu doesn't have a consistent size
> ---
>
> Key: NIFI-2788
> URL: https://issues.apache.org/jira/browse/NIFI-2788
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.0.0
> Environment: OSX, Chrome
>Reporter: Andrew Grande
>Assignee: Scott Aslan
>Priority: Trivial
> Attachments: screenshot.png
>
>
> The global menu cluster icon has incorrect size, the Cluster menu item is 
> shifted to the right as a result.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1841) SQLFeatureNotSupportedException with ExecuteSQL and Oracle Driver

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507559#comment-15507559
 ] 

ASF GitHub Bot commented on NIFI-1841:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1035


> SQLFeatureNotSupportedException with ExecuteSQL and Oracle Driver
> -
>
> Key: NIFI-1841
> URL: https://issues.apache.org/jira/browse/NIFI-1841
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Bryan Bende
>Assignee: Matt Burgess
>Priority: Minor
> Fix For: 1.1.0
>
>
> Using ExecuteSQL with the Oracle ojdbc7.jar, the following exception is 
> produced:
> {code}
> java.sql.SQLFeatureNotSupportedException: Unsupported feature 
> at oracle.jdbc.driver.ClobAccessor.getBytes(ClobAccessor.java:534) ~[na:na] 
> at 
> oracle.jdbc.driver.GeneratedStatement.getBytes(GeneratedStatement.java:151) 
> ~[na:na] 
> at 
> oracle.jdbc.driver.GeneratedScrollableResultSet.getBytes(GeneratedScrollableResultSet.java:326)
>  ~[na:na] 
> at 
> org.apache.commons.dbcp.DelegatingResultSet.getBytes(DelegatingResultSet.java:241)
>  ~[na:na] 
> at 
> org.apache.commons.dbcp.DelegatingResultSet.getBytes(DelegatingResultSet.java:241)
>  ~[na:na] 
> at 
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:105)
>  ~[nifi-standard-processors-0.6.1.jar:0.6.1] 
> at 
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:72)
>  ~[nifi-standard-processors-0.6.1.jar:0.6.1] 
> at 
> org.apache.nifi.processors.standard.ExecuteSQL$1.process(ExecuteSQL.java:158) 
> ~[nifi-standard-processors-0.6.1.jar:0.6.1] 
> {code}
> The issue appears to be related to calling ResultSet getBytes().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1841) SQLFeatureNotSupportedException with ExecuteSQL and Oracle Driver

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507556#comment-15507556
 ] 

ASF subversion and git services commented on NIFI-1841:
---

Commit 68291636cb3e823013c430ccbffbc449d178b89d in nifi's branch 
refs/heads/master from [~mattyb149]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=6829163 ]

NIFI-1841: Fixed support for CLOB/BLOB types

This closes #1035.

Signed-off-by: Bryan Bende 


> SQLFeatureNotSupportedException with ExecuteSQL and Oracle Driver
> -
>
> Key: NIFI-1841
> URL: https://issues.apache.org/jira/browse/NIFI-1841
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Bryan Bende
>Assignee: Matt Burgess
>Priority: Minor
> Fix For: 1.1.0
>
>
> Using ExecuteSQL with the Oracle ojdbc7.jar, the following exception is 
> produced:
> {code}
> java.sql.SQLFeatureNotSupportedException: Unsupported feature 
> at oracle.jdbc.driver.ClobAccessor.getBytes(ClobAccessor.java:534) ~[na:na] 
> at 
> oracle.jdbc.driver.GeneratedStatement.getBytes(GeneratedStatement.java:151) 
> ~[na:na] 
> at 
> oracle.jdbc.driver.GeneratedScrollableResultSet.getBytes(GeneratedScrollableResultSet.java:326)
>  ~[na:na] 
> at 
> org.apache.commons.dbcp.DelegatingResultSet.getBytes(DelegatingResultSet.java:241)
>  ~[na:na] 
> at 
> org.apache.commons.dbcp.DelegatingResultSet.getBytes(DelegatingResultSet.java:241)
>  ~[na:na] 
> at 
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:105)
>  ~[nifi-standard-processors-0.6.1.jar:0.6.1] 
> at 
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:72)
>  ~[nifi-standard-processors-0.6.1.jar:0.6.1] 
> at 
> org.apache.nifi.processors.standard.ExecuteSQL$1.process(ExecuteSQL.java:158) 
> ~[nifi-standard-processors-0.6.1.jar:0.6.1] 
> {code}
> The issue appears to be related to calling ResultSet getBytes().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1035: NIFI-1841: Fixed support for CLOB/BLOB types

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1035


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1040: [NIFI-2775] update Provenance table GoTo button vis...

[scottyaslan]

GitHub user scottyaslan opened a pull request:

https://github.com/apache/nifi/pull/1040

[NIFI-2775] update Provenance table GoTo button visibility in FF



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/scottyaslan/nifi NIFI-2775

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1040.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1040


commit 44ab51de2563bccd711eff87c7a880013d8d8208
Author: Scott Aslan 
Date:   2016-09-20T19:17:39Z

[NIFI-2775] update Provenance table GoTo button visibility in FF




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-2794) Expose a getOneFromEachConnection method in ProcessSession

[Pierre Villard (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pierre Villard updated NIFI-2794:
-
Attachment: image.png

The idea was to cover this kind of situation:

!image.png!

Example: I have a processor A going through a list of IDs and generating one 
FlowFile per ID. I have a processor B fetching data from one source for the 
given ID and the processor C does the same from another source. The processor D 
could use this method to get FlowFiles from both B and C for the same ID and 
join the data into one flow file.

I do realize now that this would be covered by the ongoing work with NIFI-2590, 
NIFI-1926, NIFI-2735.

There are more general patterns to cover here and this, without any doubt, can 
be achieved in a more elegant/general way.

> Expose a getOneFromEachConnection method in ProcessSession
> --
>
> Key: NIFI-2794
> URL: https://issues.apache.org/jira/browse/NIFI-2794
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 1.0.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
> Attachments: image.png
>
>
> In case a processor has multiple incoming connections, it could be 
> interesting to expose a method allowing users to get exactly one flow file 
> per incoming connection in a row.
> That could unlock opportunities such as performing join operations on 
> datasets.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2787) PersistentProvenanceRepository rollover can fail on immense indexed attributes

[Michael Moser (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507484#comment-15507484
 ] 

Michael Moser commented on NIFI-2787:
-

Using 1.1.0-SNAPSHOT the provenance rollover and index doesn't fail, presumably 
due to fixes in NIFI-2395.  But this indicates that NIFI-2429 would really be 
useful to have in 0.x.

> PersistentProvenanceRepository rollover can fail on immense indexed attributes
> --
>
> Key: NIFI-2787
> URL: https://issues.apache.org/jira/browse/NIFI-2787
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Michael Moser
>
> Accidentally created an immense attribute (36,000 bytes), which I indexed 
> with nifi.provenance.repository.indexed.attributes.  Received this error.
> ERROR [Provenance Repository Rollover Thread-1] 
> o.a.n.p.PersistentProvenanceRepository Failed to rollover Provenance 
> repository due to java.lang.IllegalArgumentException: Document contains at 
> least one immense term in field="FOO" (whose UTF8 encoding is longer than the 
> max length 32766), all of which were skipped. Please correct the analyzer to 
> not produce such terms.
> Perhaps this is as simple as changing 
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/RepositoryConfiguration.java#L37
>  to 32766 to match Lucene.  Investigation & testing needed.
> For background, this Lucene ticket made exceeding the term size limit an 
> IllegalArgumentException https://issues.apache.org/jira/browse/LUCENE-5472



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2775) UI - Go To button in Provenance table is not visible in Firefox

[Scott Aslan (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507480#comment-15507480
 ] 

Scott Aslan commented on NIFI-2775:
---

It seems there is a bug with firefox 
(https://bugzilla.mozilla.org/show_bug.cgi?id=488725) from time to time and 
looks like it is difficult to always reproduce. I think we can just wrap the 
icons in a  and I will post a PR.

> UI - Go To button in Provenance table is not visible in Firefox
> ---
>
> Key: NIFI-2775
> URL: https://issues.apache.org/jira/browse/NIFI-2775
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Scott Aslan
> Fix For: 1.1.0
>
>
> In the Provenance table, the Go To button appears to be wrapping to the next 
> line making the button not visible. Should also verify other buttons in other 
> tables. This is only happening in Firefox.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #985: NIFI-2728 - Attempt to fix travis-ci build woes

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/985


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1039: Correcting documentation for ExecuteFlumeSink

[apiri]

GitHub user apiri opened a pull request:

https://github.com/apache/nifi/pull/1039

Correcting documentation for ExecuteFlumeSink

Removing reference to removed lineage identifier header for 
ExecuteFlumeSink additional details.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apiri/incubator-nifi executeflume-doc

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1039.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1039


commit 8d2784169ac78ecfb26e4b5cbb20904d6f6c7308
Author: Aldrin Piri 
Date:   2016-09-20T17:49:18Z

Removing reference to removed lineage identifier header for 
ExecuteFlumeSink additional details.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-2781) Broken bower install

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan updated NIFI-2781:
--
Status: Patch Available  (was: In Progress)

https://github.com/apache/nifi/pull/1030

> Broken bower install
> 
>
> Key: NIFI-2781
> URL: https://issues.apache.org/jira/browse/NIFI-2781
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.0.0
>Reporter: Andrew Psaltis
>Assignee: Scott Aslan
>
> Often times when building the NiFi source, the installation of bower fails 
> and thus the whole build fails. 
> Looking at a mvn debug log, appears to show this:
> [INFO] Running 'npm install bower' in 
> /Users/apsaltis/development/forked-nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/target/frontend-working-directory
> [WARNING] npm WARN package.json mqlight@1.0.2016051011 No repository field.
> [ERROR] npm http GET https://registry.npmjs.org/bower
> [ERROR] npm http 200 https://registry.npmjs.org/bower
> [INFO] bower@1.7.9 ../../../../../../../../../node_modules/bower
> [INFO] 
> [INFO] --- frontend-maven-plugin:1.0:bower (bower-install) @ nifi-web-ui ---
> [DEBUG] Configuring mojo com.github.eirslett:frontend-maven-plugin:1.0:bower 
> from plugin realm 
> ClassRealm[plugin>com.github.eirslett:frontend-maven-plugin:1.0, parent: 
> sun.misc.Launcher$AppClassLoader@55f96302]
> [DEBUG] Configuring mojo 
> 'com.github.eirslett:frontend-maven-plugin:1.0:bower' with basic configurator 
> -->
> [DEBUG]   (f) arguments = install  
> --config.directory=/Users/apsaltis/development/forked-nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/target/nifi-web-ui-1.0.0-SNAPSHOT/assets
> [DEBUG]   (f) installDirectory = 
> /Users/apsaltis/development/forked-nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/target/frontend-working-directory
> [DEBUG]   (f) project = MavenProject: 
> org.apache.nifi:nifi-web-ui:1.0.0-SNAPSHOT @ 
> /Users/apsaltis/development/forked-nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/pom.xml
> [DEBUG]   (f) repositorySystemSession = 
> org.eclipse.aether.DefaultRepositorySystemSession@315edc50
> [DEBUG]   (f) session = org.apache.maven.execution.MavenSession@4052c8c2
> [DEBUG]   (f) skip = false
> [DEBUG]   (f) skipTests = false
> [DEBUG]   (f) workingDirectory = 
> /Users/apsaltis/development/forked-nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/target/frontend-working-directory
> [DEBUG]   (f) execution = com.github.eirslett:frontend-maven-plugin:1.0:bower 
> {execution: bower-install}
> [DEBUG] -- end configuration --
> [INFO] Running 'bower install 
> --config.directory=/Users/apsaltis/development/forked-nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/target/nifi-web-ui-1.0.0-SNAPSHOT/assets'
>  in 
> /Users/apsaltis/development/forked-nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/target/frontend-working-directory
> [ERROR] 
> [ERROR] module.js:340
> [ERROR] throw err;
> [ERROR]   ^
> [ERROR] Error: Cannot find module 
> '/Users/apsaltis/development/forked-nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/target/frontend-working-directory/node_modules/bower/bin/bower'
> [ERROR] at Function.Module._resolveFilename (module.js:338:15)
> [ERROR] at Function.Module._load (module.js:280:25)
> [ERROR] at Function.Module.runMain (module.js:497:10)
> [ERROR] at startup (node.js:119:16)
> [ERROR] at node.js:901:3
> [INFO] 
> 
> In essence bower is not installed. This is reproducible in my environment, 
> which looks like this:
> $ printenv
> TERM_PROGRAM=iTerm.app
> TERM=xterm
> SHELL=/bin/bash
> TMPDIR=/var/folders/5l/xlzh00b10_lf16cd06dqwvs4gn/T/
> Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.jdxbKLHhHZ/Render
> TERM_PROGRAM_VERSION=3.0.7
> OLDPWD=/Users/apsaltis/development/forked-nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui
> TERM_SESSION_ID=w0t0p1:4401C1E6-AF27-43E2-91BF-7814B628BEC1
> USER=apsaltis
> SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.VHh1dBoww3/Listeners
> __CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0
> PATH=/Library/Java/JavaVirtualMachines/jdk1.8.0_77.jdk/Contents/Home/bin:/Library/Frameworks/Python.framework/Versions/2.7/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sb...
> PWD=/Users/apsaltis/development/forked-nifi
> JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_77.jdk/Contents/Home
> LANG=en_US.UTF-8
> ITERM_PROFILE=Default
> XPC_FLAGS=0x0
> XPC_SERVICE_NAME=0
> SHLVL=1
> HOME=/Users/apsaltis
> COLORFGBG=7;0
> 

[jira] [Assigned] (NIFI-2775) UI - Go To button in Provenance table is not visible in Firefox

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan reassigned NIFI-2775:
-

Assignee: Scott Aslan

> UI - Go To button in Provenance table is not visible in Firefox
> ---
>
> Key: NIFI-2775
> URL: https://issues.apache.org/jira/browse/NIFI-2775
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Scott Aslan
> Fix For: 1.1.0
>
>
> In the Provenance table, the Go To button appears to be wrapping to the next 
> line making the button not visible. Should also verify other buttons in other 
> tables. This is only happening in Firefox.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2797) Authorization header not submitted when clicking Download from Templates window

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-2797:
--
Status: Patch Available  (was: In Progress)

> Authorization header not submitted when clicking Download from Templates 
> window
> ---
>
> Key: NIFI-2797
> URL: https://issues.apache.org/jira/browse/NIFI-2797
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Scott Wagner
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> I am running on a standalone instance of Apache NiFi.  It is configured to 
> use a local LDAP server for authentication, and I am logging in as a user 
> with full permissions.
> When browsing the templates, and I click on the "Download" link, a new tab is 
> opened in the browser but the error message of {{Unable to perform the 
> desired action due to insufficient permissions. Contact the system 
> administrator.}}
> Checking the link that is submitted via developer tools, I noticed that the 
> Authorization header is not being submitted.  If I use curl to get the URL 
> that the browser is trying to get but submit an Authorization header for my 
> valid session, I am able to download the template XML.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2797) Authorization header not submitted when clicking Download from Templates window

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507108#comment-15507108
 ] 

ASF GitHub Bot commented on NIFI-2797:
--

GitHub user mcgilman opened a pull request:

https://github.com/apache/nifi/pull/1038

Correcting download URIs for OTPs

NIFI-2797:
- Correcting download URIs for OTPs.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mcgilman/nifi NIFI-2797

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1038.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1038


commit 9194527bc9a0231ea8caeb9b2e64a453bdff2ac2
Author: Matt Gilman 
Date:   2016-09-20T16:50:55Z

NIFI-2797:
- Correcting download URIs for OTPs.




> Authorization header not submitted when clicking Download from Templates 
> window
> ---
>
> Key: NIFI-2797
> URL: https://issues.apache.org/jira/browse/NIFI-2797
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Scott Wagner
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> I am running on a standalone instance of Apache NiFi.  It is configured to 
> use a local LDAP server for authentication, and I am logging in as a user 
> with full permissions.
> When browsing the templates, and I click on the "Download" link, a new tab is 
> opened in the browser but the error message of {{Unable to perform the 
> desired action due to insufficient permissions. Contact the system 
> administrator.}}
> Checking the link that is submitted via developer tools, I noticed that the 
> Authorization header is not being submitted.  If I use curl to get the URL 
> that the browser is trying to get but submit an Authorization header for my 
> valid session, I am able to download the template XML.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1038: Correcting download URIs for OTPs

[mcgilman]

GitHub user mcgilman opened a pull request:

https://github.com/apache/nifi/pull/1038

Correcting download URIs for OTPs

NIFI-2797:
- Correcting download URIs for OTPs.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mcgilman/nifi NIFI-2797

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1038.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1038


commit 9194527bc9a0231ea8caeb9b2e64a453bdff2ac2
Author: Matt Gilman 
Date:   2016-09-20T16:50:55Z

NIFI-2797:
- Correcting download URIs for OTPs.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507100#comment-15507100
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79656833
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
--- End diff --

Lately the conventional wisdom for the name() method is to choose a 
machine-friendly name (like "validate-json-schema-file") and use displayName() 
for the user-friendly name. This will help when we do more work with 
internationalization. Not a requirement, just a suggestion :)


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": 

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507099#comment-15507099
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79657612
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
+.description("The path to the Schema file that is to be used 
for validation")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully validated 
against the schema are routed to this relationship")
+.build();
+public static final Relationship REL_INVALID = new 
Relationship.Builder()
+.name("invalid")
+.description("FlowFiles that are not valid according to the 
specified schema are routed to this relationship")
+.build();
+
+private List properties;
+private Set relationships;
+private final AtomicReference schemaRef = new 
AtomicReference<>();
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+final List properties = new ArrayList<>();
+properties.add(SCHEMA_FILE);
--- End diff --

For NiFi clusters, it may not 

[GitHub] nifi pull request #1037: NIFI-1893 Add processor for validating JSON

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79656833
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
--- End diff --

Lately the conventional wisdom for the name() method is to choose a 
machine-friendly name (like "validate-json-schema-file") and use displayName() 
for the user-friendly name. This will help when we do more work with 
internationalization. Not a requirement, just a suggestion :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1037: NIFI-1893 Add processor for validating JSON

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79657612
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
+.description("The path to the Schema file that is to be used 
for validation")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully validated 
against the schema are routed to this relationship")
+.build();
+public static final Relationship REL_INVALID = new 
Relationship.Builder()
+.name("invalid")
+.description("FlowFiles that are not valid according to the 
specified schema are routed to this relationship")
+.build();
+
+private List properties;
+private Set relationships;
+private final AtomicReference schemaRef = new 
AtomicReference<>();
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+final List properties = new ArrayList<>();
+properties.add(SCHEMA_FILE);
--- End diff --

For NiFi clusters, it may not be prudent (or as easy as desired) to have 
the schema file available on all the nodes. However this is a still a good 
property to have. Perhaps also there could be a Schema Body property, where the 
schema can be pasted in? That might 

[GitHub] nifi pull request #1037: NIFI-1893 Add processor for validating JSON

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79657850
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
+.description("The path to the Schema file that is to be used 
for validation")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully validated 
against the schema are routed to this relationship")
+.build();
+public static final Relationship REL_INVALID = new 
Relationship.Builder()
+.name("invalid")
+.description("FlowFiles that are not valid according to the 
specified schema are routed to this relationship")
+.build();
+
+private List properties;
+private Set relationships;
+private final AtomicReference schemaRef = new 
AtomicReference<>();
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+final List properties = new ArrayList<>();
+properties.add(SCHEMA_FILE);
+this.properties = Collections.unmodifiableList(properties);
+
+final Set relationships = new HashSet<>();
+relationships.add(REL_VALID);
+relationships.add(REL_INVALID);
+this.relationships = 

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507082#comment-15507082
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79654106
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
--- End diff --

Could you use both ``.name()`` and ``.displayName()`` as explained here:

https://mail-archives.apache.org/mod_mbox/nifi-dev/201605.mbox/%3c5a6fdf1e-1889-46fe-a3c4-5d2f0a905...@apache.org%3E


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> 

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507083#comment-15507083
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79655327
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
+.description("The path to the Schema file that is to be used 
for validation")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully validated 
against the schema are routed to this relationship")
+.build();
+public static final Relationship REL_INVALID = new 
Relationship.Builder()
+.name("invalid")
+.description("FlowFiles that are not valid according to the 
specified schema are routed to this relationship")
+.build();
+
+private List properties;
+private Set relationships;
+private final AtomicReference schemaRef = new 
AtomicReference<>();
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+final List properties = new ArrayList<>();
+properties.add(SCHEMA_FILE);
+this.properties = 

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507084#comment-15507084
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79654708
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
--- End diff --

Would it be possible to have this property to accept both a file and a 
string representation of the schema? If the input given by the user is an 
existing file we use a file as you propose, otherwise we use the string as the 
schema definition. I think that a lot of users would like to pass a schema 
definition without using a file (given that, in cluster mode, the file will 
have to be available on each node). Thoughts?


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", 

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507081#comment-15507081
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79655788
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
+.description("The path to the Schema file that is to be used 
for validation")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully validated 
against the schema are routed to this relationship")
+.build();
+public static final Relationship REL_INVALID = new 
Relationship.Builder()
+.name("invalid")
+.description("FlowFiles that are not valid according to the 
specified schema are routed to this relationship")
+.build();
+
+private List properties;
+private Set relationships;
+private final AtomicReference schemaRef = new 
AtomicReference<>();
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+final List properties = new ArrayList<>();
+properties.add(SCHEMA_FILE);
+this.properties = 

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507080#comment-15507080
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79655764
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
+.description("The path to the Schema file that is to be used 
for validation")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully validated 
against the schema are routed to this relationship")
+.build();
+public static final Relationship REL_INVALID = new 
Relationship.Builder()
+.name("invalid")
+.description("FlowFiles that are not valid according to the 
specified schema are routed to this relationship")
+.build();
+
+private List properties;
+private Set relationships;
+private final AtomicReference schemaRef = new 
AtomicReference<>();
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+final List properties = new ArrayList<>();
+properties.add(SCHEMA_FILE);
+this.properties = 

[GitHub] nifi pull request #1037: NIFI-1893 Add processor for validating JSON

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79654708
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
--- End diff --

Would it be possible to have this property to accept both a file and a 
string representation of the schema? If the input given by the user is an 
existing file we use a file as you propose, otherwise we use the string as the 
schema definition. I think that a lot of users would like to pass a schema 
definition without using a file (given that, in cluster mode, the file will 
have to be available on each node). Thoughts?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1037: NIFI-1893 Add processor for validating JSON

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79655788
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
+.description("The path to the Schema file that is to be used 
for validation")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully validated 
against the schema are routed to this relationship")
+.build();
+public static final Relationship REL_INVALID = new 
Relationship.Builder()
+.name("invalid")
+.description("FlowFiles that are not valid according to the 
specified schema are routed to this relationship")
+.build();
+
+private List properties;
+private Set relationships;
+private final AtomicReference schemaRef = new 
AtomicReference<>();
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+final List properties = new ArrayList<>();
+properties.add(SCHEMA_FILE);
+this.properties = Collections.unmodifiableList(properties);
+
+final Set relationships = new HashSet<>();
+relationships.add(REL_VALID);
+relationships.add(REL_INVALID);
+this.relationships = 

[GitHub] nifi pull request #1037: NIFI-1893 Add processor for validating JSON

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79655327
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
+.description("The path to the Schema file that is to be used 
for validation")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully validated 
against the schema are routed to this relationship")
+.build();
+public static final Relationship REL_INVALID = new 
Relationship.Builder()
+.name("invalid")
+.description("FlowFiles that are not valid according to the 
specified schema are routed to this relationship")
+.build();
+
+private List properties;
+private Set relationships;
+private final AtomicReference schemaRef = new 
AtomicReference<>();
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+final List properties = new ArrayList<>();
+properties.add(SCHEMA_FILE);
+this.properties = Collections.unmodifiableList(properties);
+
+final Set relationships = new HashSet<>();
+relationships.add(REL_VALID);
+relationships.add(REL_INVALID);
+this.relationships = 

[GitHub] nifi pull request #1037: NIFI-1893 Add processor for validating JSON

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79655764
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
+.description("The path to the Schema file that is to be used 
for validation")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully validated 
against the schema are routed to this relationship")
+.build();
+public static final Relationship REL_INVALID = new 
Relationship.Builder()
+.name("invalid")
+.description("FlowFiles that are not valid according to the 
specified schema are routed to this relationship")
+.build();
+
+private List properties;
+private Set relationships;
+private final AtomicReference schemaRef = new 
AtomicReference<>();
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+final List properties = new ArrayList<>();
+properties.add(SCHEMA_FILE);
+this.properties = Collections.unmodifiableList(properties);
+
+final Set relationships = new HashSet<>();
+relationships.add(REL_VALID);
+relationships.add(REL_INVALID);
+this.relationships = 

[GitHub] nifi pull request #1037: NIFI-1893 Add processor for validating JSON

[pvillard31]

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79654106
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
--- End diff --

Could you use both ``.name()`` and ``.displayName()`` as explained here:

https://mail-archives.apache.org/mod_mbox/nifi-dev/201605.mbox/%3c5a6fdf1e-1889-46fe-a3c4-5d2f0a905...@apache.org%3E


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2794) Expose a getOneFromEachConnection method in ProcessSession

[Oleg Zhurakousky (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507041#comment-15507041
 ] 

Oleg Zhurakousky commented on NIFI-2794:


[~pvillard] I've read the description, yet I am still not getting as to what is 
the value for this. I am definitely missing something, but may be you can 
clarify a bit more. . . may be a more concrete use case that it would address? 
I know you mentioned join, but there is no guarantee that "all required" will 
arrive at the pre-defined time, right? Anyway, help me out here ;)

> Expose a getOneFromEachConnection method in ProcessSession
> --
>
> Key: NIFI-2794
> URL: https://issues.apache.org/jira/browse/NIFI-2794
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 1.0.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
>
> In case a processor has multiple incoming connections, it could be 
> interesting to expose a method allowing users to get exactly one flow file 
> per incoming connection in a row.
> That could unlock opportunities such as performing join operations on 
> datasets.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2797) Authorization header not submitted when clicking Download from Templates window

[Matt Gilman (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507032#comment-15507032
 ] 

Matt Gilman commented on NIFI-2797:
---

Someone was just submitted this same issue on the mailing list. Here's some 
additional details...

{noformat}
When using an authentication which will use the API tokens, download requests 
are processed using a one-time password token (since they become part of the 
URL). These are only honored for certain endpoints which do not appear correct. 

As a work-around, you could use clients certificates, download via a curl 
command, or use View as it is not subject to the same endpoint check (when not 
clustered).
{noformat}

> Authorization header not submitted when clicking Download from Templates 
> window
> ---
>
> Key: NIFI-2797
> URL: https://issues.apache.org/jira/browse/NIFI-2797
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Scott Wagner
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> I am running on a standalone instance of Apache NiFi.  It is configured to 
> use a local LDAP server for authentication, and I am logging in as a user 
> with full permissions.
> When browsing the templates, and I click on the "Download" link, a new tab is 
> opened in the browser but the error message of {{Unable to perform the 
> desired action due to insufficient permissions. Contact the system 
> administrator.}}
> Checking the link that is submitted via developer tools, I noticed that the 
> Authorization header is not being submitted.  If I use curl to get the URL 
> that the browser is trying to get but submit an Authorization header for my 
> valid session, I am able to download the template XML.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507030#comment-15507030
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

GitHub user bartoszjkwozniak opened a pull request:

https://github.com/apache/nifi/pull/1037

NIFI-1893 Add processor for validating JSON

Signed-off-by: Bartosz Wozniak 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bartoszjkwozniak/nifi NIFI-1893

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1037.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1037


commit 554d15fa994b74db67e2906de31a8fe001045d95
Author: Bartosz Wozniak 
Date:   2016-09-19T11:37:54Z

NIFI-1893 Add processor for validating JSON

Signed-off-by: Bartosz Wozniak 




> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (NIFI-2797) Authorization header not submitted when clicking Download from Templates window

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman reassigned NIFI-2797:
-

Assignee: Matt Gilman

> Authorization header not submitted when clicking Download from Templates 
> window
> ---
>
> Key: NIFI-2797
> URL: https://issues.apache.org/jira/browse/NIFI-2797
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Scott Wagner
>Assignee: Matt Gilman
> Fix For: 1.1.0
>
>
> I am running on a standalone instance of Apache NiFi.  It is configured to 
> use a local LDAP server for authentication, and I am logging in as a user 
> with full permissions.
> When browsing the templates, and I click on the "Download" link, a new tab is 
> opened in the browser but the error message of {{Unable to perform the 
> desired action due to insufficient permissions. Contact the system 
> administrator.}}
> Checking the link that is submitted via developer tools, I noticed that the 
> Authorization header is not being submitted.  If I use curl to get the URL 
> that the browser is trying to get but submit an Authorization header for my 
> valid session, I am able to download the template XML.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2797) Authorization header not submitted when clicking Download from Templates window

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-2797:
--
Fix Version/s: 1.1.0

> Authorization header not submitted when clicking Download from Templates 
> window
> ---
>
> Key: NIFI-2797
> URL: https://issues.apache.org/jira/browse/NIFI-2797
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Scott Wagner
> Fix For: 1.1.0
>
>
> I am running on a standalone instance of Apache NiFi.  It is configured to 
> use a local LDAP server for authentication, and I am logging in as a user 
> with full permissions.
> When browsing the templates, and I click on the "Download" link, a new tab is 
> opened in the browser but the error message of {{Unable to perform the 
> desired action due to insufficient permissions. Contact the system 
> administrator.}}
> Checking the link that is submitted via developer tools, I noticed that the 
> Authorization header is not being submitted.  If I use curl to get the URL 
> that the browser is trying to get but submit an Authorization header for my 
> valid session, I am able to download the template XML.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2774) ConsumeJMS processor losses messages on NiFi restart

[Oleg Zhurakousky (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507018#comment-15507018
 ] 

Oleg Zhurakousky commented on NIFI-2774:


Added new PR https://github.com/apache/nifi/pull/1036

> ConsumeJMS processor losses messages on NiFi restart
> 
>
> Key: NIFI-2774
> URL: https://issues.apache.org/jira/browse/NIFI-2774
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Christopher McDermott
>Assignee: Oleg Zhurakousky
>Priority: Critical
> Fix For: 1.1.0, 0.8.0
>
> Attachments: 2774.patch
>
>
> ConsumeJMS processor uses auto-acknowledge mode.  Unlike the deprecated 
> GetJMSQueue processor it does not provide a way to specify a different ACK 
> mode (i.e. client-acknowledge.)  Using auto-acknowledge, acknowledges message 
> receipt from JMS *before* the messages are actually added to the flow.  This 
> leads to data-loss on NiFi stop (or crash.)
> I believe the fix for this is to allow the user to specify the ACK mode in 
> the processor configuration like is allowed by the GetJMSQueue processor.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2774) ConsumeJMS processor losses messages on NiFi restart

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507016#comment-15507016
 ] 

ASF GitHub Bot commented on NIFI-2774:
--

GitHub user olegz opened a pull request:

https://github.com/apache/nifi/pull/1036

NIFI-2774 added configurable QoS options



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/olegz/nifi NIFI-2774B

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1036.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1036


commit 5df326d886cb806d345468cd08ecd34f5b85cdd8
Author: Oleg Zhurakousky 
Date:   2016-09-20T14:49:22Z

NIFI-2774 added configurable QoS options




> ConsumeJMS processor losses messages on NiFi restart
> 
>
> Key: NIFI-2774
> URL: https://issues.apache.org/jira/browse/NIFI-2774
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Christopher McDermott
>Assignee: Oleg Zhurakousky
>Priority: Critical
> Fix For: 1.1.0, 0.8.0
>
> Attachments: 2774.patch
>
>
> ConsumeJMS processor uses auto-acknowledge mode.  Unlike the deprecated 
> GetJMSQueue processor it does not provide a way to specify a different ACK 
> mode (i.e. client-acknowledge.)  Using auto-acknowledge, acknowledges message 
> receipt from JMS *before* the messages are actually added to the flow.  This 
> leads to data-loss on NiFi stop (or crash.)
> I believe the fix for this is to allow the user to specify the ACK mode in 
> the processor configuration like is allowed by the GetJMSQueue processor.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1036: NIFI-2774 added configurable QoS options

[olegz]

GitHub user olegz opened a pull request:

https://github.com/apache/nifi/pull/1036

NIFI-2774 added configurable QoS options



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/olegz/nifi NIFI-2774B

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1036.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1036


commit 5df326d886cb806d345468cd08ecd34f5b85cdd8
Author: Oleg Zhurakousky 
Date:   2016-09-20T14:49:22Z

NIFI-2774 added configurable QoS options




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #986: Nifi 2381

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/986


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (NIFI-2797) Authorization header not submitted when clicking Download from Templates window

[Scott Wagner (JIRA)]

Scott Wagner created NIFI-2797:
--

 Summary: Authorization header not submitted when clicking Download 
from Templates window
 Key: NIFI-2797
 URL: https://issues.apache.org/jira/browse/NIFI-2797
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core UI
Affects Versions: 1.0.0
Reporter: Scott Wagner


I am running on a standalone instance of Apache NiFi.  It is configured to use 
a local LDAP server for authentication, and I am logging in as a user with full 
permissions.

When browsing the templates, and I click on the "Download" link, a new tab is 
opened in the browser but the error message of {{Unable to perform the desired 
action due to insufficient permissions. Contact the system administrator.}}

Checking the link that is submitted via developer tools, I noticed that the 
Authorization header is not being submitted.  If I use curl to get the URL that 
the browser is trying to get but submit an Authorization header for my valid 
session, I am able to download the template XML.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2774) ConsumeJMS processor losses messages on NiFi restart

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15506906#comment-15506906
 ] 

ASF GitHub Bot commented on NIFI-2774:
--

Github user olegz commented on the issue:

https://github.com/apache/nifi/pull/1022
  
Closing this PR because of the complexity with fixing merge conflicts due 
to changes that came in with #1027. Will submit new one shortly.


> ConsumeJMS processor losses messages on NiFi restart
> 
>
> Key: NIFI-2774
> URL: https://issues.apache.org/jira/browse/NIFI-2774
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Christopher McDermott
>Assignee: Oleg Zhurakousky
>Priority: Critical
> Fix For: 1.1.0, 0.8.0
>
> Attachments: 2774.patch
>
>
> ConsumeJMS processor uses auto-acknowledge mode.  Unlike the deprecated 
> GetJMSQueue processor it does not provide a way to specify a different ACK 
> mode (i.e. client-acknowledge.)  Using auto-acknowledge, acknowledges message 
> receipt from JMS *before* the messages are actually added to the flow.  This 
> leads to data-loss on NiFi stop (or crash.)
> I believe the fix for this is to allow the user to specify the ACK mode in 
> the processor configuration like is allowed by the GetJMSQueue processor.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-1841) SQLFeatureNotSupportedException with ExecuteSQL and Oracle Driver

[Matt Burgess (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess updated NIFI-1841:
---
Fix Version/s: 1.1.0
   Status: Patch Available  (was: In Progress)

> SQLFeatureNotSupportedException with ExecuteSQL and Oracle Driver
> -
>
> Key: NIFI-1841
> URL: https://issues.apache.org/jira/browse/NIFI-1841
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Bryan Bende
>Assignee: Matt Burgess
>Priority: Minor
> Fix For: 1.1.0
>
>
> Using ExecuteSQL with the Oracle ojdbc7.jar, the following exception is 
> produced:
> {code}
> java.sql.SQLFeatureNotSupportedException: Unsupported feature 
> at oracle.jdbc.driver.ClobAccessor.getBytes(ClobAccessor.java:534) ~[na:na] 
> at 
> oracle.jdbc.driver.GeneratedStatement.getBytes(GeneratedStatement.java:151) 
> ~[na:na] 
> at 
> oracle.jdbc.driver.GeneratedScrollableResultSet.getBytes(GeneratedScrollableResultSet.java:326)
>  ~[na:na] 
> at 
> org.apache.commons.dbcp.DelegatingResultSet.getBytes(DelegatingResultSet.java:241)
>  ~[na:na] 
> at 
> org.apache.commons.dbcp.DelegatingResultSet.getBytes(DelegatingResultSet.java:241)
>  ~[na:na] 
> at 
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:105)
>  ~[nifi-standard-processors-0.6.1.jar:0.6.1] 
> at 
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:72)
>  ~[nifi-standard-processors-0.6.1.jar:0.6.1] 
> at 
> org.apache.nifi.processors.standard.ExecuteSQL$1.process(ExecuteSQL.java:158) 
> ~[nifi-standard-processors-0.6.1.jar:0.6.1] 
> {code}
> The issue appears to be related to calling ResultSet getBytes().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1841) SQLFeatureNotSupportedException with ExecuteSQL and Oracle Driver

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15506909#comment-15506909
 ] 

ASF GitHub Bot commented on NIFI-1841:
--

GitHub user mattyb149 opened a pull request:

https://github.com/apache/nifi/pull/1035

NIFI-1841: Fixed support for CLOB/BLOB types



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mattyb149/nifi NIFI-1841

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1035.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1035


commit 784ddca8e58e329264543d7c8e925f81d383fbdd
Author: Matt Burgess 
Date:   2016-09-20T15:35:53Z

NIFI-1841: Fixed support for CLOB/BLOB types




> SQLFeatureNotSupportedException with ExecuteSQL and Oracle Driver
> -
>
> Key: NIFI-1841
> URL: https://issues.apache.org/jira/browse/NIFI-1841
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Bryan Bende
>Assignee: Matt Burgess
>Priority: Minor
>
> Using ExecuteSQL with the Oracle ojdbc7.jar, the following exception is 
> produced:
> {code}
> java.sql.SQLFeatureNotSupportedException: Unsupported feature 
> at oracle.jdbc.driver.ClobAccessor.getBytes(ClobAccessor.java:534) ~[na:na] 
> at 
> oracle.jdbc.driver.GeneratedStatement.getBytes(GeneratedStatement.java:151) 
> ~[na:na] 
> at 
> oracle.jdbc.driver.GeneratedScrollableResultSet.getBytes(GeneratedScrollableResultSet.java:326)
>  ~[na:na] 
> at 
> org.apache.commons.dbcp.DelegatingResultSet.getBytes(DelegatingResultSet.java:241)
>  ~[na:na] 
> at 
> org.apache.commons.dbcp.DelegatingResultSet.getBytes(DelegatingResultSet.java:241)
>  ~[na:na] 
> at 
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:105)
>  ~[nifi-standard-processors-0.6.1.jar:0.6.1] 
> at 
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:72)
>  ~[nifi-standard-processors-0.6.1.jar:0.6.1] 
> at 
> org.apache.nifi.processors.standard.ExecuteSQL$1.process(ExecuteSQL.java:158) 
> ~[nifi-standard-processors-0.6.1.jar:0.6.1] 
> {code}
> The issue appears to be related to calling ResultSet getBytes().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1035: NIFI-1841: Fixed support for CLOB/BLOB types

[mattyb149]

GitHub user mattyb149 opened a pull request:

https://github.com/apache/nifi/pull/1035

NIFI-1841: Fixed support for CLOB/BLOB types



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mattyb149/nifi NIFI-1841

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1035.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1035


commit 784ddca8e58e329264543d7c8e925f81d383fbdd
Author: Matt Burgess 
Date:   2016-09-20T15:35:53Z

NIFI-1841: Fixed support for CLOB/BLOB types




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2774) ConsumeJMS processor losses messages on NiFi restart

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15506907#comment-15506907
 ] 

ASF GitHub Bot commented on NIFI-2774:
--

Github user olegz closed the pull request at:

https://github.com/apache/nifi/pull/1022


> ConsumeJMS processor losses messages on NiFi restart
> 
>
> Key: NIFI-2774
> URL: https://issues.apache.org/jira/browse/NIFI-2774
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Christopher McDermott
>Assignee: Oleg Zhurakousky
>Priority: Critical
> Fix For: 1.1.0, 0.8.0
>
> Attachments: 2774.patch
>
>
> ConsumeJMS processor uses auto-acknowledge mode.  Unlike the deprecated 
> GetJMSQueue processor it does not provide a way to specify a different ACK 
> mode (i.e. client-acknowledge.)  Using auto-acknowledge, acknowledges message 
> receipt from JMS *before* the messages are actually added to the flow.  This 
> leads to data-loss on NiFi stop (or crash.)
> I believe the fix for this is to allow the user to specify the ACK mode in 
> the processor configuration like is allowed by the GetJMSQueue processor.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1022: NIFI-2774 changed the default ACK mode to CLIENT

[olegz]

Github user olegz commented on the issue:

https://github.com/apache/nifi/pull/1022
  
Closing this PR because of the complexity with fixing merge conflicts due 
to changes that came in with #1027. Will submit new one shortly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1022: NIFI-2774 changed the default ACK mode to CLIENT

[olegz]

Github user olegz closed the pull request at:

https://github.com/apache/nifi/pull/1022


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1034: Allow Content Type to be set in PutS3Object process...

[vegaed]

GitHub user vegaed opened a pull request:

https://github.com/apache/nifi/pull/1034

Allow Content Type to be set in PutS3Object processor



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vegaed/nifi master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1034.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1034


commit 3901afe0619ae4568a13fb7b4a071d1e7ef97f82
Author: Edgardo 
Date:   2016-09-20T15:08:39Z

Allow Content Type to be set in PutS3Object processor




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (NIFI-1841) SQLFeatureNotSupportedException with ExecuteSQL and Oracle Driver

[Matt Burgess (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess reassigned NIFI-1841:
--

Assignee: Matt Burgess

> SQLFeatureNotSupportedException with ExecuteSQL and Oracle Driver
> -
>
> Key: NIFI-1841
> URL: https://issues.apache.org/jira/browse/NIFI-1841
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Bryan Bende
>Assignee: Matt Burgess
>Priority: Minor
>
> Using ExecuteSQL with the Oracle ojdbc7.jar, the following exception is 
> produced:
> {code}
> java.sql.SQLFeatureNotSupportedException: Unsupported feature 
> at oracle.jdbc.driver.ClobAccessor.getBytes(ClobAccessor.java:534) ~[na:na] 
> at 
> oracle.jdbc.driver.GeneratedStatement.getBytes(GeneratedStatement.java:151) 
> ~[na:na] 
> at 
> oracle.jdbc.driver.GeneratedScrollableResultSet.getBytes(GeneratedScrollableResultSet.java:326)
>  ~[na:na] 
> at 
> org.apache.commons.dbcp.DelegatingResultSet.getBytes(DelegatingResultSet.java:241)
>  ~[na:na] 
> at 
> org.apache.commons.dbcp.DelegatingResultSet.getBytes(DelegatingResultSet.java:241)
>  ~[na:na] 
> at 
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:105)
>  ~[nifi-standard-processors-0.6.1.jar:0.6.1] 
> at 
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:72)
>  ~[nifi-standard-processors-0.6.1.jar:0.6.1] 
> at 
> org.apache.nifi.processors.standard.ExecuteSQL$1.process(ExecuteSQL.java:158) 
> ~[nifi-standard-processors-0.6.1.jar:0.6.1] 
> {code}
> The issue appears to be related to calling ResultSet getBytes().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (NIFI-2790) Set JMS destination name on send/receive instead of using the default destination

[Oleg Zhurakousky (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Zhurakousky resolved NIFI-2790.

Resolution: Fixed

[~jfrazee] All is merged! Thank you for the contribution!

> Set JMS destination name on send/receive instead of using the default 
> destination
> -
>
> Key: NIFI-2790
> URL: https://issues.apache.org/jira/browse/NIFI-2790
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Joey Frazee
>Priority: Minor
> Fix For: 1.1.0, 0.8.0
>
>
> ConsumeJMS and PublishJMS currently pull their destination name from the 
> default JMS destination (setDefaultDestinationName() on the JmsTemplate). The 
> effect this has is that attribute expressions are evaluated with respect to 
> the context only and not the FlowFile, so expression language support really 
> only extends to EL functions and variables from the variable registry.
> This doesn't have a big impact on ConsumeJMS since it doesn't take input, but 
> it means that destinations can be set at runtime in PublishJMS.
> The JmsTemplate send() and receive() can take the destination name as an 
> argument though, so these method variants should be used so EL support is 
> fully enabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-1214) Mock Framework should allow order-independent assumptions on FlowFiles

[Joseph Percivall (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-1214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Percivall updated NIFI-1214:
---
   Resolution: Fixed
Fix Version/s: 1.1.0
   Status: Resolved  (was: Patch Available)

Resoled via PR #1033

With the following commit: 

https://github.com/apache/nifi/commit/ad3d63d204c1e4408c91728d4c5d76a75e14c8f9

> Mock Framework should allow order-independent assumptions on FlowFiles
> --
>
> Key: NIFI-1214
> URL: https://issues.apache.org/jira/browse/NIFI-1214
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Tools and Build
>Reporter: Mark Payne
>Assignee: Toivo Adams
> Fix For: 1.1.0
>
>
> A common pattern in unit testing is to iterate over all FlowFiles that are 
> output to a Relationship and verify that each FlowFile matches one criteria 
> or another and that all criteria are met. For example, the following code 
> snippet from TestRouteText  verifies that two FlowFiles were output and that 
> Criteria A was met by one of them and Criteria B was met by the other:
> {code}
> final List list = 
> runner.getFlowFilesForRelationship("o");
> boolean found1 = false;
> boolean found2 = false;
> for (final MockFlowFile mff : list) {
> if (mff.getAttribute(RouteText.GROUP_ATTRIBUTE_KEY).equals("1")) {
> mff.assertContentEquals("1,hello\n1,good-bye");
> found1 = true;
> } else {
> mff.assertAttributeEquals(RouteText.GROUP_ATTRIBUTE_KEY, "2");
> mff.assertContentEquals("2,world\n");
> found2 = true;
> }
> }
> assertTrue(found1);
> assertTrue(found2);
> {code}
> This is very verbose, and error-prone. It could be done much more concisely 
> if we have a method like:
> {code}
> TestRunner.assertAllConditionsMet( Relationship relationship, 
> FlowFileVerifier... verifier );
> {code}
> Where FlowFileVerifier is able to verify some condition on a FlowFile. This 
> method would then be responsible for ensuring that each FlowFile that was 
> routed to 'relationship' meets one of the criteria specified by a verifier, 
> and that all of the verifiers were met. For example:
> {code}
> runner.assertAllConditionsMet( "o", 
> { mff -> mff.isAttributeEqual(RouteText.GROUP_ATTRIBUTE_KEY, "1") && 
> mff.isContentEqual("1,hello\n1,good-bye") },
> { mff -> mff.isAttributeEqual(RouteText.GROUP_ATTRIBUTE_KEY, "2") && 
> mff.isContentEqual("2,world\n") }
> );
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1033: nifi-1214d Mock Framework …on FlowFiles. This rep...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1033


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-1870) Restore go to source/destination capability

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-1870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-1870:
--
Status: Patch Available  (was: In Progress)

> Restore go to source/destination capability
> ---
>
> Key: NIFI-1870
> URL: https://issues.apache.org/jira/browse/NIFI-1870
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Minor
> Fix For: 1.1.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (NIFI-2469) UI - Component Run Status

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman resolved NIFI-2469.
---
   Resolution: Fixed
Fix Version/s: (was: 1.1.0)
   1.0.0

Addressed in NIFI-2545 by ensuring that we reporting an active thread when 
STARTING and STOPPING.

> UI - Component Run Status
> -
>
> Key: NIFI-2469
> URL: https://issues.apache.org/jira/browse/NIFI-2469
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Reporter: Matt Gilman
> Fix For: 1.0.0
>
>
> With the introduction of the Starting/Stopping scheduled states, the UI can 
> be confusing when representing the current run state of a component.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2775) UI - Go To button in Provenance table is not visible in Firefox

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-2775:
--
Fix Version/s: (was: 1.0.0)
   1.1.0

> UI - Go To button in Provenance table is not visible in Firefox
> ---
>
> Key: NIFI-2775
> URL: https://issues.apache.org/jira/browse/NIFI-2775
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Gilman
> Fix For: 1.1.0
>
>
> In the Provenance table, the Go To button appears to be wrapping to the next 
> line making the button not visible. Should also verify other buttons in other 
> tables. This is only happening in Firefox.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (NIFI-2789) Add JMS properties to FlowFile attributes on receive in ConsumeJMS

[Oleg Zhurakousky (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Zhurakousky resolved NIFI-2789.

Resolution: Fixed

The PR submitted to this JIRA was closed since its changes are incorporated 
into pr 1027 NIFI-2790

> Add JMS properties to FlowFile attributes on receive in ConsumeJMS
> --
>
> Key: NIFI-2789
> URL: https://issues.apache.org/jira/browse/NIFI-2789
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Joey Frazee
>Priority: Minor
> Fix For: 1.1.0, 0.8.0
>
>
> ConsumeJMS currently adds JMS headers to the FlowFile attributes when it 
> receives a message but ignores any JMS properties coming through. It should 
> be reading both the headers and properties and merging them into the FlowFile 
> attributes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1033: nifi-1214d Mock Framework …on FlowFiles. This replaces p...

[JPercivall]

Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/1033
  
+1

Visually verified code and did a contrib check build. Looks good 
@ToivoAdams, thanks! I will merge it in.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (NIFI-2775) UI - Go To button in Provenance table is not visible in Firefox

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman resolved NIFI-2775.
---
   Resolution: Fixed
Fix Version/s: (was: 1.1.0)
   1.0.0

Issue was resolved in NIFI-2545 by ensuring that we show active threads when 
STARTING and STOPPING.

> UI - Go To button in Provenance table is not visible in Firefox
> ---
>
> Key: NIFI-2775
> URL: https://issues.apache.org/jira/browse/NIFI-2775
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Gilman
> Fix For: 1.0.0
>
>
> In the Provenance table, the Go To button appears to be wrapping to the next 
> line making the button not visible. Should also verify other buttons in other 
> tables. This is only happening in Firefox.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (NIFI-2775) UI - Go To button in Provenance table is not visible in Firefox

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman reopened NIFI-2775:
---

Incorrectly closed wrong JIRA. Doh.

> UI - Go To button in Provenance table is not visible in Firefox
> ---
>
> Key: NIFI-2775
> URL: https://issues.apache.org/jira/browse/NIFI-2775
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Reporter: Matt Gilman
> Fix For: 1.0.0
>
>
> In the Provenance table, the Go To button appears to be wrapping to the next 
> line making the button not visible. Should also verify other buttons in other 
> tables. This is only happening in Firefox.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2789) Add JMS properties to FlowFile attributes on receive in ConsumeJMS

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15506701#comment-15506701
 ] 

ASF subversion and git services commented on NIFI-2789:
---

Commit 58fdfdc76e2e59bffbbcdd4e96ad2a190b412b29 in nifi's branch refs/heads/0.x 
from [~jfrazee]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=58fdfdc ]

NIFI-2789, NIFI-2790 - Read JMS properties and add to FlowFile attributes in 
ConsumeJMS

Remove unused assertEquals import

Move destination from default to send/receive to support EL better


> Add JMS properties to FlowFile attributes on receive in ConsumeJMS
> --
>
> Key: NIFI-2789
> URL: https://issues.apache.org/jira/browse/NIFI-2789
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Joey Frazee
>Priority: Minor
> Fix For: 1.1.0
>
>
> ConsumeJMS currently adds JMS headers to the FlowFile attributes when it 
> receives a message but ignores any JMS properties coming through. It should 
> be reading both the headers and properties and merging them into the FlowFile 
> attributes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2790) Set JMS destination name on send/receive instead of using the default destination

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15506704#comment-15506704
 ] 

ASF subversion and git services commented on NIFI-2790:
---

Commit c2e98f96ebe30a6a8c919ed8a747a5829f5dea0f in nifi's branch refs/heads/0.x 
from [~ozhurakousky]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=c2e98f9 ]

NIFI-2789, NIFI-2790 - polishing


> Set JMS destination name on send/receive instead of using the default 
> destination
> -
>
> Key: NIFI-2790
> URL: https://issues.apache.org/jira/browse/NIFI-2790
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Joey Frazee
>Priority: Minor
> Fix For: 1.1.0
>
>
> ConsumeJMS and PublishJMS currently pull their destination name from the 
> default JMS destination (setDefaultDestinationName() on the JmsTemplate). The 
> effect this has is that attribute expressions are evaluated with respect to 
> the context only and not the FlowFile, so expression language support really 
> only extends to EL functions and variables from the variable registry.
> This doesn't have a big impact on ConsumeJMS since it doesn't take input, but 
> it means that destinations can be set at runtime in PublishJMS.
> The JmsTemplate send() and receive() can take the destination name as an 
> argument though, so these method variants should be used so EL support is 
> fully enabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2790) Set JMS destination name on send/receive instead of using the default destination

[Oleg Zhurakousky (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Zhurakousky updated NIFI-2790:
---
Fix Version/s: 0.8.0

> Set JMS destination name on send/receive instead of using the default 
> destination
> -
>
> Key: NIFI-2790
> URL: https://issues.apache.org/jira/browse/NIFI-2790
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Joey Frazee
>Priority: Minor
> Fix For: 1.1.0, 0.8.0
>
>
> ConsumeJMS and PublishJMS currently pull their destination name from the 
> default JMS destination (setDefaultDestinationName() on the JmsTemplate). The 
> effect this has is that attribute expressions are evaluated with respect to 
> the context only and not the FlowFile, so expression language support really 
> only extends to EL functions and variables from the variable registry.
> This doesn't have a big impact on ConsumeJMS since it doesn't take input, but 
> it means that destinations can be set at runtime in PublishJMS.
> The JmsTemplate send() and receive() can take the destination name as an 
> argument though, so these method variants should be used so EL support is 
> fully enabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2789) Add JMS properties to FlowFile attributes on receive in ConsumeJMS

[Oleg Zhurakousky (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Zhurakousky updated NIFI-2789:
---
Fix Version/s: 0.8.0

> Add JMS properties to FlowFile attributes on receive in ConsumeJMS
> --
>
> Key: NIFI-2789
> URL: https://issues.apache.org/jira/browse/NIFI-2789
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Joey Frazee
>Priority: Minor
> Fix For: 1.1.0, 0.8.0
>
>
> ConsumeJMS currently adds JMS headers to the FlowFile attributes when it 
> receives a message but ignores any JMS properties coming through. It should 
> be reading both the headers and properties and merging them into the FlowFile 
> attributes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2789) Add JMS properties to FlowFile attributes on receive in ConsumeJMS

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15506703#comment-15506703
 ] 

ASF subversion and git services commented on NIFI-2789:
---

Commit c2e98f96ebe30a6a8c919ed8a747a5829f5dea0f in nifi's branch refs/heads/0.x 
from [~ozhurakousky]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=c2e98f9 ]

NIFI-2789, NIFI-2790 - polishing


> Add JMS properties to FlowFile attributes on receive in ConsumeJMS
> --
>
> Key: NIFI-2789
> URL: https://issues.apache.org/jira/browse/NIFI-2789
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Joey Frazee
>Priority: Minor
> Fix For: 1.1.0
>
>
> ConsumeJMS currently adds JMS headers to the FlowFile attributes when it 
> receives a message but ignores any JMS properties coming through. It should 
> be reading both the headers and properties and merging them into the FlowFile 
> attributes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2790) Set JMS destination name on send/receive instead of using the default destination

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15506702#comment-15506702
 ] 

ASF subversion and git services commented on NIFI-2790:
---

Commit 58fdfdc76e2e59bffbbcdd4e96ad2a190b412b29 in nifi's branch refs/heads/0.x 
from [~jfrazee]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=58fdfdc ]

NIFI-2789, NIFI-2790 - Read JMS properties and add to FlowFile attributes in 
ConsumeJMS

Remove unused assertEquals import

Move destination from default to send/receive to support EL better


> Set JMS destination name on send/receive instead of using the default 
> destination
> -
>
> Key: NIFI-2790
> URL: https://issues.apache.org/jira/browse/NIFI-2790
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Joey Frazee
>Priority: Minor
> Fix For: 1.1.0
>
>
> ConsumeJMS and PublishJMS currently pull their destination name from the 
> default JMS destination (setDefaultDestinationName() on the JmsTemplate). The 
> effect this has is that attribute expressions are evaluated with respect to 
> the context only and not the FlowFile, so expression language support really 
> only extends to EL functions and variables from the variable registry.
> This doesn't have a big impact on ConsumeJMS since it doesn't take input, but 
> it means that destinations can be set at runtime in PublishJMS.
> The JmsTemplate send() and receive() can take the destination name as an 
> argument though, so these method variants should be used so EL support is 
> fully enabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (NIFI-2796) Rest Api Docs - Cluster summary endpoint documentation is wrong

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-2796:
--
Fix Version/s: 1.1.0

> Rest Api Docs - Cluster summary endpoint documentation is wrong
> ---
>
> Key: NIFI-2796
> URL: https://issues.apache.org/jira/browse/NIFI-2796
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework, Documentation & Website
>Reporter: Matt Gilman
> Fix For: 1.1.0
>
>
> Documentation for the cluster summary endpoint should indicate that it 
> returns a ClusterSummaryEntity.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (NIFI-2796) Rest Api Docs - Cluster summary endpoint documentation is wrong

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-2796:
-

 Summary: Rest Api Docs - Cluster summary endpoint documentation is 
wrong
 Key: NIFI-2796
 URL: https://issues.apache.org/jira/browse/NIFI-2796
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework, Documentation & Website
Reporter: Matt Gilman


Documentation for the cluster summary endpoint should indicate that it returns 
a ClusterSummaryEntity.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #803: nifi-1214c Mock Framework should allow order-independent as...

[JPercivall]

Github user JPercivall commented on the issue:

https://github.com/apache/nifi/pull/803
  
@ToivoAdams yup, it's complaining because when you "rebase" you are 
re-writing history. When you try to push it's out of sync with the branch 
you're pushing to and it tells you to "pull" in order to be coordinated. By 
doing a "force push", you're saying "forget whatever the current state of the 
branch is, what I'm pushing is correct".

This is very nice when you're doing it to your own branch/PR but it's 
pretty self-explanatory how bad it could be doing to the Apache repo.

I'm gonna take a look at the new PR now.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request #33: Minifi 108

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/33#discussion_r79616293
  
--- Diff: 
minifi-commons/minifi-commons-schema/src/main/java/org/apache/nifi/minifi/commons/schema/ConfigSchema.java
 ---
@@ -198,4 +305,31 @@ public ComponentStatusRepositorySchema 
getComponentStatusRepositoryProperties()
 public ProvenanceRepositorySchema getProvenanceRepositorySchema() {
 return provenanceRepositorySchema;
 }
+
+public int getVersion() {
+return CONFIG_VERSION;
+}
+
+/**
+ * Will replace all characters not in [A-Za-z0-9_] with _
+ * 
+ * This has potential for collisions so it will also append numbers as 
necessary to prevent that
+ *
+ * @param ids  id map of already incremented numbers
+ * @param name the name
+ * @return a unique filesystem-friendly id
+ */
+protected static String getUniqueId(Map ids, String 
name) {
+String baseId = StringUtil.isNullOrEmpty(name) ? EMPTY_NAME : 
ID_REPLACE_PATTERN.matcher(name).replaceAll("_");
+String id = baseId;
+Integer idNum = ids.get(baseId);
+while (ids.containsKey(id)) {
+id = baseId + "_" + idNum++;
+}
+if (id != baseId) {
--- End diff --

Totally down with "!=" for this case.

Lets just comment why it's a valid case so people aren't confused later on.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2790) Set JMS destination name on send/receive instead of using the default destination

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15506541#comment-15506541
 ] 

ASF subversion and git services commented on NIFI-2790:
---

Commit c23867605857f8380140ba4d69437b58585e9cba in nifi's branch 
refs/heads/master from [~jfrazee]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=c238676 ]

NIFI-2789, NIFI-2790 - Read JMS properties and add to FlowFile attributes in 
ConsumeJMS

Remove unused assertEquals import

Move destination from default to send/receive to support EL better


> Set JMS destination name on send/receive instead of using the default 
> destination
> -
>
> Key: NIFI-2790
> URL: https://issues.apache.org/jira/browse/NIFI-2790
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Joey Frazee
>Priority: Minor
> Fix For: 1.1.0
>
>
> ConsumeJMS and PublishJMS currently pull their destination name from the 
> default JMS destination (setDefaultDestinationName() on the JmsTemplate). The 
> effect this has is that attribute expressions are evaluated with respect to 
> the context only and not the FlowFile, so expression language support really 
> only extends to EL functions and variables from the variable registry.
> This doesn't have a big impact on ConsumeJMS since it doesn't take input, but 
> it means that destinations can be set at runtime in PublishJMS.
> The JmsTemplate send() and receive() can take the destination name as an 
> argument though, so these method variants should be used so EL support is 
> fully enabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2789) Add JMS properties to FlowFile attributes on receive in ConsumeJMS

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15506540#comment-15506540
 ] 

ASF subversion and git services commented on NIFI-2789:
---

Commit c23867605857f8380140ba4d69437b58585e9cba in nifi's branch 
refs/heads/master from [~jfrazee]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=c238676 ]

NIFI-2789, NIFI-2790 - Read JMS properties and add to FlowFile attributes in 
ConsumeJMS

Remove unused assertEquals import

Move destination from default to send/receive to support EL better


> Add JMS properties to FlowFile attributes on receive in ConsumeJMS
> --
>
> Key: NIFI-2789
> URL: https://issues.apache.org/jira/browse/NIFI-2789
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Joey Frazee
>Priority: Minor
> Fix For: 1.1.0
>
>
> ConsumeJMS currently adds JMS headers to the FlowFile attributes when it 
> receives a message but ignores any JMS properties coming through. It should 
> be reading both the headers and properties and merging them into the FlowFile 
> attributes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2790) Set JMS destination name on send/receive instead of using the default destination

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15506545#comment-15506545
 ] 

ASF GitHub Bot commented on NIFI-2790:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1027


> Set JMS destination name on send/receive instead of using the default 
> destination
> -
>
> Key: NIFI-2790
> URL: https://issues.apache.org/jira/browse/NIFI-2790
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Joey Frazee
>Priority: Minor
> Fix For: 1.1.0
>
>
> ConsumeJMS and PublishJMS currently pull their destination name from the 
> default JMS destination (setDefaultDestinationName() on the JmsTemplate). The 
> effect this has is that attribute expressions are evaluated with respect to 
> the context only and not the FlowFile, so expression language support really 
> only extends to EL functions and variables from the variable registry.
> This doesn't have a big impact on ConsumeJMS since it doesn't take input, but 
> it means that destinations can be set at runtime in PublishJMS.
> The JmsTemplate send() and receive() can take the destination name as an 
> argument though, so these method variants should be used so EL support is 
> fully enabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi-minifi pull request #33: Minifi 108

[brosander]

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/33#discussion_r79610484
  
--- Diff: minifi-docs/src/main/markdown/System_Admin_Guide.md ---
@@ -174,10 +174,11 @@ for example "10 sec" or "10 MB", not simply "10".
 
 The first section of config.yml is for naming and commenting on the file.
 
- Property | Description
-  | ---
-name  | The name of the file.
-comment   | A comment describing the usage of this config file.
+ Property | Description
+- | ---
+MiNiFi Config Version | The version of the configuration file.
--- End diff --

@JPercivall good point, I will try to clarify


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1027: NIFI-2790 Set JMS destination name on send/receive ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1027


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi pull request #33: Minifi 108

[brosander]

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/33#discussion_r79610232
  
--- Diff: 
minifi-commons/minifi-commons-schema/src/main/java/org/apache/nifi/minifi/commons/schema/ConfigSchema.java
 ---
@@ -96,51 +100,154 @@ public ConfigSchema(Map map) {
 addIssuesIfNotNull(provenanceReportingProperties);
 addIssuesIfNotNull(provenanceRepositorySchema);
 
+Set processorIds = new HashSet<>();
 if (processors != null) {
-
checkForDuplicateNames(FOUND_THE_FOLLOWING_DUPLICATE_PROCESSOR_NAMES, 
processors.stream().map(ProcessorSchema::getName).collect(Collectors.toList()));
+List processorIdList = 
processors.stream().map(ProcessorSchema::getId).collect(Collectors.toList());
+checkForDuplicates(this::addValidationIssue, 
FOUND_THE_FOLLOWING_DUPLICATE_PROCESSOR_IDS, processorIdList);
 for (ProcessorSchema processorSchema : processors) {
 addIssuesIfNotNull(processorSchema);
 }
+processorIds.addAll(processorIdList);
 }
 
 if (connections != null) {
-
checkForDuplicateNames(FOUND_THE_FOLLOWING_DUPLICATE_CONNECTION_NAMES, 
connections.stream().map(ConnectionSchema::getName).collect(Collectors.toList()));
+List idList = 
connections.stream().map(ConnectionSchema::getId).filter(s -> 
!StringUtil.isNullOrEmpty(s)).collect(Collectors.toList());
+checkForDuplicates(this::addValidationIssue, 
FOUND_THE_FOLLOWING_DUPLICATE_CONNECTION_IDS, idList);
 for (ConnectionSchema connectionSchema : connections) {
 addIssuesIfNotNull(connectionSchema);
 }
 }
 
+Set remoteInputPortIds = new HashSet<>();
 if (remoteProcessingGroups != null) {
-
checkForDuplicateNames(FOUND_THE_FOLLOWING_DUPLICATE_REMOTE_PROCESSING_GROUP_NAMES,
 
remoteProcessingGroups.stream().map(RemoteProcessingGroupSchema::getName).collect(Collectors.toList()));
+checkForDuplicates(this::addValidationIssue, 
FOUND_THE_FOLLOWING_DUPLICATE_REMOTE_PROCESSING_GROUP_NAMES,
+
remoteProcessingGroups.stream().map(RemoteProcessingGroupSchema::getName).collect(Collectors.toList()));
 for (RemoteProcessingGroupSchema remoteProcessingGroupSchema : 
remoteProcessingGroups) {
 addIssuesIfNotNull(remoteProcessingGroupSchema);
 }
+List remoteProcessingGroups = 
getRemoteProcessingGroups();
+if (remoteProcessingGroups != null) {
+List remoteInputPortIdList = 
remoteProcessingGroups.stream().filter(r -> r.getInputPorts() != null)
+.flatMap(r -> 
r.getInputPorts().stream()).map(RemoteInputPortSchema::getId).collect(Collectors.toList());
+checkForDuplicates(this::addValidationIssue, 
FOUND_THE_FOLLOWING_DUPLICATE_REMOTE_INPUT_PORT_IDS, remoteInputPortIdList);
+remoteInputPortIds.addAll(remoteInputPortIdList);
+}
+}
+
+Set duplicateIds = new HashSet<>(processorIds);
+duplicateIds.retainAll(remoteInputPortIds);
+if (duplicateIds.size() > 0) {
+addValidationIssue(FOUND_THE_FOLLOWING_DUPLICATE_IDS + 
duplicateIds.stream().sorted().collect(Collectors.joining(", ")));
+}
+}
+
+protected List getProcessorSchemas(List 
processorMaps) {
+if (processorMaps == null) {
+return null;
 }
+List processors = 
convertListToType(processorMaps, "processor", ProcessorSchema.class, 
PROCESSORS_KEY);
+
+Map idMap = 
processors.stream().map(ProcessorSchema::getId).filter(
+s -> 
!StringUtil.isNullOrEmpty(s)).collect(Collectors.toMap(Function.identity(), s 
-> 2, Integer::compareTo));
+
+// Set unset ids
+processors.stream().filter(connection -> 
StringUtil.isNullOrEmpty(connection.getId())).forEachOrdered(processor -> 
processor.setId(getUniqueId(idMap, processor.getName(;
+
+return processors;
 }
 
-private void checkForDuplicateNames(String errorMessagePrefix, 
List names) {
-if (names != null) {
-Set seenNames = new HashSet<>();
-Set duplicateNames = new TreeSet<>();
-for (String name : names) {
-if (!seenNames.add(name)) {
-duplicateNames.add(name);
+protected List getConnectionSchemas(List 
connectionMaps) {
+if (connectionMaps == null) {
+return null;
+}
+List connections = 
convertListToType(connectionMaps, "connection", 

[GitHub] nifi-minifi pull request #33: Minifi 108

[brosander]

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi-minifi/pull/33#discussion_r79609970
  
--- Diff: 
minifi-commons/minifi-commons-schema/src/main/java/org/apache/nifi/minifi/commons/schema/ConfigSchema.java
 ---
@@ -198,4 +305,31 @@ public ComponentStatusRepositorySchema 
getComponentStatusRepositoryProperties()
 public ProvenanceRepositorySchema getProvenanceRepositorySchema() {
 return provenanceRepositorySchema;
 }
+
+public int getVersion() {
+return CONFIG_VERSION;
+}
+
+/**
+ * Will replace all characters not in [A-Za-z0-9_] with _
+ * 
+ * This has potential for collisions so it will also append numbers as 
necessary to prevent that
+ *
+ * @param ids  id map of already incremented numbers
+ * @param name the name
+ * @return a unique filesystem-friendly id
+ */
+protected static String getUniqueId(Map ids, String 
name) {
+String baseId = StringUtil.isNullOrEmpty(name) ? EMPTY_NAME : 
ID_REPLACE_PATTERN.matcher(name).replaceAll("_");
+String id = baseId;
+Integer idNum = ids.get(baseId);
+while (ids.containsKey(id)) {
+id = baseId + "_" + idNum++;
+}
+if (id != baseId) {
--- End diff --

@JPercivall I'm aware that most of the time this signals a bug, the 
assignment of id to baseId is only a few lines up, I'm just trying to see if it 
was reassigned in the while loop, meaning the reference equals should (and 
seems to based on testing) do the trick.

I can change to a boolean flag if you think it would make things clearer.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (NIFI-2790) Set JMS destination name on send/receive instead of using the default destination

[Joey Frazee (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joey Frazee reassigned NIFI-2790:
-

Assignee: Joey Frazee  (was: Oleg Zhurakousky)

> Set JMS destination name on send/receive instead of using the default 
> destination
> -
>
> Key: NIFI-2790
> URL: https://issues.apache.org/jira/browse/NIFI-2790
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Joey Frazee
>Priority: Minor
> Fix For: 1.1.0
>
>
> ConsumeJMS and PublishJMS currently pull their destination name from the 
> default JMS destination (setDefaultDestinationName() on the JmsTemplate). The 
> effect this has is that attribute expressions are evaluated with respect to 
> the context only and not the FlowFile, so expression language support really 
> only extends to EL functions and variables from the variable registry.
> This doesn't have a big impact on ConsumeJMS since it doesn't take input, but 
> it means that destinations can be set at runtime in PublishJMS.
> The JmsTemplate send() and receive() can take the destination name as an 
> argument though, so these method variants should be used so EL support is 
> fully enabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2789) Add JMS properties to FlowFile attributes on receive in ConsumeJMS

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15506509#comment-15506509
 ] 

ASF GitHub Bot commented on NIFI-2789:
--

Github user jfrazee closed the pull request at:

https://github.com/apache/nifi/pull/1026


> Add JMS properties to FlowFile attributes on receive in ConsumeJMS
> --
>
> Key: NIFI-2789
> URL: https://issues.apache.org/jira/browse/NIFI-2789
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Joey Frazee
>Priority: Minor
> Fix For: 1.1.0
>
>
> ConsumeJMS currently adds JMS headers to the FlowFile attributes when it 
> receives a message but ignores any JMS properties coming through. It should 
> be reading both the headers and properties and merging them into the FlowFile 
> attributes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #1026: NIFI-2789 Add JMS properties to FlowFile attributes on rec...

[jfrazee]

Github user jfrazee commented on the issue:

https://github.com/apache/nifi/pull/1026
  
Closing in favor of #1027 which includes the properties enhancement too.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2789) Add JMS properties to FlowFile attributes on receive in ConsumeJMS

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15506510#comment-15506510
 ] 

ASF GitHub Bot commented on NIFI-2789:
--

Github user jfrazee commented on the issue:

https://github.com/apache/nifi/pull/1026
  
Closing in favor of #1027 which includes the properties enhancement too.


> Add JMS properties to FlowFile attributes on receive in ConsumeJMS
> --
>
> Key: NIFI-2789
> URL: https://issues.apache.org/jira/browse/NIFI-2789
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Joey Frazee
>Priority: Minor
> Fix For: 1.1.0
>
>
> ConsumeJMS currently adds JMS headers to the FlowFile attributes when it 
> receives a message but ignores any JMS properties coming through. It should 
> be reading both the headers and properties and merging them into the FlowFile 
> attributes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi pull request #1026: NIFI-2789 Add JMS properties to FlowFile attributes...

[jfrazee]

Github user jfrazee closed the pull request at:

https://github.com/apache/nifi/pull/1026


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (NIFI-2789) Add JMS properties to FlowFile attributes on receive in ConsumeJMS

[Joey Frazee (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joey Frazee reassigned NIFI-2789:
-

Assignee: Joey Frazee  (was: Oleg Zhurakousky)

> Add JMS properties to FlowFile attributes on receive in ConsumeJMS
> --
>
> Key: NIFI-2789
> URL: https://issues.apache.org/jira/browse/NIFI-2789
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joey Frazee
>Assignee: Joey Frazee
>Priority: Minor
> Fix For: 1.1.0
>
>
> ConsumeJMS currently adds JMS headers to the FlowFile attributes when it 
> receives a message but ignores any JMS properties coming through. It should 
> be reading both the headers and properties and merging them into the FlowFile 
> attributes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] nifi issue #803: nifi-1214c Mock Framework should allow order-independent as...

[ToivoAdams]

Github user ToivoAdams commented on the issue:

https://github.com/apache/nifi/pull/803
  
New PR #1033

I did rebase as usual.
But git refused to push and recommended pull before push.
I did pull but it seems I should force push instead of pull.

Thanks
Toivo


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1033: nifi-1214d Mock Framework …on FlowFiles. This rep...

[ToivoAdams]

GitHub user ToivoAdams opened a pull request:

https://github.com/apache/nifi/pull/1033

nifi-1214d Mock Framework …on FlowFiles. This replaces previous nifi-1214c

New Pull Request, replaces previous #803 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ToivoAdams/nifi nifi-1214d

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1033.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1033


commit 639df94ed2282c0efc11e0249b94c9e739c3aef5
Author: Toivo Adams 
Date:   2016-09-20T08:35:07Z

nifi-1214d Mock Framework …on FlowFiles. This replaces previous nifi-1214c




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---