[jira] [Updated] (NIFI-5090) When moving files, FetchFTP and FetchSFTP should be able to dynamically create a directory if not found

[Koji Kawamura (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Koji Kawamura updated NIFI-5090:

Summary: When moving files, FetchFTP and FetchSFTP should be able to 
dynamically create a directory if not found  (was: When moving files, FetchSFTP 
should be able to dynamically create a directory if not found)

> When moving files, FetchFTP and FetchSFTP should be able to dynamically 
> create a directory if not found
> ---
>
> Key: NIFI-5090
> URL: https://issues.apache.org/jira/browse/NIFI-5090
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>Priority: Major
>
> When using FetchFTP or FetchSFTP with "Completion Strategy" "Move" files it 
> allows you to specify a directory that files are moved to.
> The directory to move files to has to exist for the files to be moved there. 
> But it will be useful if fetch FTP processors create a directory using some 
> attribute data, so that processed files are moved into a folder called:
> "processed/${year}/${month}/${day}"
> Currently with this format, the system creates an exception "FileNotFound" 
> which actually appears to be referring to the destination directory not being 
> found.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (NIFI-5090) When moving files, FetchSFTP should be able to dynamically create a directory if not found

[Koji Kawamura (JIRA)]

Koji Kawamura created NIFI-5090:
---

 Summary: When moving files, FetchSFTP should be able to 
dynamically create a directory if not found
 Key: NIFI-5090
 URL: https://issues.apache.org/jira/browse/NIFI-5090
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Extensions
Reporter: Koji Kawamura
Assignee: Koji Kawamura


When using FetchFTP or FetchSFTP with "Completion Strategy" "Move" files it 
allows you to specify a directory that files are moved to.

The directory to move files to has to exist for the files to be moved there. 
But it will be useful if fetch FTP processors create a directory using some 
attribute data, so that processed files are moved into a folder called:
"processed/${year}/${month}/${day}"

Currently with this format, the system creates an exception "FileNotFound" 
which actually appears to be referring to the destination directory not being 
found.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (NIFIREG-163) Add JDK requirement to Admin Guide

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFIREG-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan closed NIFIREG-163.
---
   Resolution: Fixed
Fix Version/s: 0.2.0

> Add JDK requirement to Admin Guide
> --
>
> Key: NIFIREG-163
> URL: https://issues.apache.org/jira/browse/NIFIREG-163
> Project: NiFi Registry
>  Issue Type: Improvement
>Affects Versions: 0.1.0
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Minor
> Fix For: 0.2.0
>
>
> Should notify users of the JDK requirement and potential error that results 
> as discussed in NIFIREG-142



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (NIFIREG-165) Changing the destination of a relation is not identified as "Local change" by NiFi registry

[Rahul Soni (JIRA)]

Rahul Soni created NIFIREG-165:
--

 Summary: Changing the destination of a relation is not identified 
as "Local change" by NiFi registry
 Key: NIFIREG-165
 URL: https://issues.apache.org/jira/browse/NIFIREG-165
 Project: NiFi Registry
  Issue Type: Bug
Affects Versions: 0.1.0
Reporter: Rahul Soni


When the destination of a relation is changed, the NiFi relation does not track 
it is a local change.

In a sample scenario, I have the flow as GenerateFF -> UpdateAttribute1 with 
UpdateAttribute2 processor hanging alone.

After I commit the flow to NiFi registry, and then change the flow to direct 
the output of GenerateFF processor to UpdateAttribute2, it is not identified as 
a change by the registry. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (NIFIREG-164) Addition of a new Processor Group Variable not identified as "Local change"

[Rahul Soni (JIRA)]

Rahul Soni created NIFIREG-164:
--

 Summary: Addition of a new Processor Group Variable not identified 
as "Local change"
 Key: NIFIREG-164
 URL: https://issues.apache.org/jira/browse/NIFIREG-164
 Project: NiFi Registry
  Issue Type: Bug
Affects Versions: 0.1.0
Reporter: Rahul Soni


Addition of a new Processor Group variable is not identified as "Local Change" 
in NiFi 1.5.0 with NiFi Registry 0.1.0.

Follows the description and issue reproduction steps.
 # Create a processor group.
 # Add some processors to the Processor Group
 # Start version control
 # Commit the flow to a new bucket
 # Add a new variable to the processor group. This will not be identified as a 
"Local change"
 # Change the position of a processor from your processor group. This will be 
identified as "Local change"
 # "Show local changes" will display only "Position changed for the processor" 
as the change
 # Import the Processor group from the registry to some other canvas.
 # The variable can be seen there.

So while committing, the variable is not identified as a change but is actually 
written to the registry.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-5089) Exclude Maven download progress output from Travis builds

[Matt Burgess (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess updated NIFI-5089:
---
Status: Patch Available  (was: In Progress)

> Exclude Maven download progress output from Travis builds
> -
>
> Key: NIFI-5089
> URL: https://issues.apache.org/jira/browse/NIFI-5089
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Tools and Build
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
>
> The current Travis jobs are failing due to the logs being too long, and it 
> seems to be caused by Maven outputting consecutive "Progress" lines to show a 
> dependency being downloaded. We should exclude these lines from the output in 
> order to allow the Travis builds to succeed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5089) Exclude Maven download progress output from Travis builds

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441515#comment-16441515
 ] 

ASF GitHub Bot commented on NIFI-5089:
--

GitHub user mattyb149 opened a pull request:

https://github.com/apache/nifi/pull/2641

NIFI-5089: Exclude Maven progress output from Travis builds

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mattyb149/nifi NIFI-5089

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2641.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2641


commit 9c1fe041064ef639ee85626c9e567f37e8b9b37d
Author: Matthew Burgess 
Date:   2018-04-17T21:16:28Z

NIFI-5089: Exclude Maven progress output from Travis builds




> Exclude Maven download progress output from Travis builds
> -
>
> Key: NIFI-5089
> URL: https://issues.apache.org/jira/browse/NIFI-5089
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Tools and Build
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
>
> The current Travis jobs are failing due to the logs being too long, and it 
> seems to be caused by Maven outputting consecutive "Progress" lines to show a 
> dependency being downloaded. We should exclude these lines from the output in 
> order to allow the Travis builds to succeed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2641: NIFI-5089: Exclude Maven progress output from Travi...

[mattyb149]

GitHub user mattyb149 opened a pull request:

https://github.com/apache/nifi/pull/2641

NIFI-5089: Exclude Maven progress output from Travis builds

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mattyb149/nifi NIFI-5089

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2641.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2641


commit 9c1fe041064ef639ee85626c9e567f37e8b9b37d
Author: Matthew Burgess 
Date:   2018-04-17T21:16:28Z

NIFI-5089: Exclude Maven progress output from Travis builds




---

[jira] [Assigned] (NIFI-5089) Exclude Maven download progress output from Travis builds

[Matt Burgess (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess reassigned NIFI-5089:
--

Assignee: Matt Burgess

> Exclude Maven download progress output from Travis builds
> -
>
> Key: NIFI-5089
> URL: https://issues.apache.org/jira/browse/NIFI-5089
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Tools and Build
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
>
> The current Travis jobs are failing due to the logs being too long, and it 
> seems to be caused by Maven outputting consecutive "Progress" lines to show a 
> dependency being downloaded. We should exclude these lines from the output in 
> order to allow the Travis builds to succeed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (NIFI-5089) Exclude Maven download progress output from Travis builds

[Matt Burgess (JIRA)]

Matt Burgess created NIFI-5089:
--

 Summary: Exclude Maven download progress output from Travis builds
 Key: NIFI-5089
 URL: https://issues.apache.org/jira/browse/NIFI-5089
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Tools and Build
Reporter: Matt Burgess


The current Travis jobs are failing due to the logs being too long, and it 
seems to be caused by Maven outputting consecutive "Progress" lines to show a 
dependency being downloaded. We should exclude these lines from the output in 
order to allow the Travis builds to succeed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-2359) Improve AttributesToJSON so it can output booleans, numbers and datetimes

[Matt Burgess (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441474#comment-16441474
 ] 

Matt Burgess commented on NIFI-2359:


What about the case where you only want to convert some strings into numbers 
but not all of them, such as zip codes?

There is a workaround if you know which attributes will be written to JSON: 
After AttributesToJSON you can use ConvertRecord with a JsonTreeReader (with a 
schema specifying all fields as strings) and a JsonRecordSetWriter (with the 
desired output schema).

> Improve AttributesToJSON so it can output booleans, numbers and datetimes
> -
>
> Key: NIFI-2359
> URL: https://issues.apache.org/jira/browse/NIFI-2359
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 0.7.0
>Reporter: Tristan Morris
>Priority: Minor
>  Labels: beginner
> Attachments: 
> 0001-NIFI-2359-Added-support-to-AttributesToJSON-to-conve.patch
>
>
> The AttributesToJSON processor only outputs string content 
> {
>"isAwesome": "true"
> }
> Rather than 
> {
>   "isAwesome": true
> }
> It would be useful for AttributesToJSON to output json native types like 
> booleans, numbers and datetimes



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFIREG-163) Add JDK requirement to Admin Guide

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFIREG-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441405#comment-16441405
 ] 

ASF GitHub Bot commented on NIFIREG-163:


Github user asfgit closed the pull request at:

https://github.com/apache/nifi-registry/pull/113


> Add JDK requirement to Admin Guide
> --
>
> Key: NIFIREG-163
> URL: https://issues.apache.org/jira/browse/NIFIREG-163
> Project: NiFi Registry
>  Issue Type: Improvement
>Affects Versions: 0.1.0
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Minor
>
> Should notify users of the JDK requirement and potential error that results 
> as discussed in NIFIREG-142



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-registry pull request #113: NIFIREG-163 Added JDK information to System...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi-registry/pull/113


---

[jira] [Resolved] (NIFI-4995) Release Apache NiFi 1.6.0

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt resolved NIFI-4995.
---
Resolution: Fixed

> Release Apache NiFi 1.6.0
> -
>
> Key: NIFI-4995
> URL: https://issues.apache.org/jira/browse/NIFI-4995
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Tools and Build
>Affects Versions: 1.6.0
>Reporter: Joseph Witt
>Assignee: Joseph Witt
>Priority: Blocker
> Fix For: 1.6.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (MINIFICPP-458) Network management controller service for support ipv6

[bqiu (JIRA)]

bqiu created MINIFICPP-458:
--

 Summary: Network management controller service for support ipv6
 Key: MINIFICPP-458
 URL: https://issues.apache.org/jira/browse/MINIFICPP-458
 Project: NiFi MiNiFi C++
  Issue Type: Improvement
Reporter: bqiu
Assignee: bqiu






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (MINIFICPP-457) Network management controller service for interface binding for socket

[bqiu (JIRA)]

bqiu created MINIFICPP-457:
--

 Summary: Network management controller service for interface 
binding for socket
 Key: MINIFICPP-457
 URL: https://issues.apache.org/jira/browse/MINIFICPP-457
 Project: NiFi MiNiFi C++
  Issue Type: Improvement
Reporter: bqiu
Assignee: bqiu


Network management controller service for interface binding for socket



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4456) Update JSON Record Reader / Writer to allow for 'json per line' format

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441283#comment-16441283
 ] 

ASF GitHub Bot commented on NIFI-4456:
--

GitHub user mattyb149 opened a pull request:

https://github.com/apache/nifi/pull/2640

NIFI-4456: Support multiple JSON objects in JSON record reader/writers

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [x] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mattyb149/nifi NIFI-4456

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2640.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2640


commit 16ca567ea25e931679ca6f81ff020a94ab7be32d
Author: Matthew Burgess 
Date:   2018-04-17T15:04:56Z

NIFI-4456: Support multiple JSON objects in JSON record reader/writers




> Update JSON Record Reader / Writer to allow for 'json per line' format
> --
>
> Key: NIFI-4456
> URL: https://issues.apache.org/jira/browse/NIFI-4456
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Matt Burgess
>Priority: Major
>
> It is common, especially for archiving purposes, to have many JSON objects 
> combined with new-lines in between, in order to delimit the records. It would 
> be useful to allow record readers and writers to support this, instead of 
> requiring that JSON records being elements in a JSON Array.
> For example, the following JSON Is considered two records:
> {code}
> [
>   { "greeting" : "hello", "id" : 1 },
>   { "greeting" : "good-bye", "id" : 2 }
> ]
> {code}
> It would be beneficial to also support the format:
> {code}
> { "greeting" : "hello", "id" : 1 }
> { "greeting" : "good-bye", "id" : 2 }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-4456) Update JSON Record Reader / Writer to allow for 'json per line' format

[Matt Burgess (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess updated NIFI-4456:
---
Status: Patch Available  (was: In Progress)

> Update JSON Record Reader / Writer to allow for 'json per line' format
> --
>
> Key: NIFI-4456
> URL: https://issues.apache.org/jira/browse/NIFI-4456
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Matt Burgess
>Priority: Major
>
> It is common, especially for archiving purposes, to have many JSON objects 
> combined with new-lines in between, in order to delimit the records. It would 
> be useful to allow record readers and writers to support this, instead of 
> requiring that JSON records being elements in a JSON Array.
> For example, the following JSON Is considered two records:
> {code}
> [
>   { "greeting" : "hello", "id" : 1 },
>   { "greeting" : "good-bye", "id" : 2 }
> ]
> {code}
> It would be beneficial to also support the format:
> {code}
> { "greeting" : "hello", "id" : 1 }
> { "greeting" : "good-bye", "id" : 2 }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2640: NIFI-4456: Support multiple JSON objects in JSON re...

[mattyb149]

GitHub user mattyb149 opened a pull request:

https://github.com/apache/nifi/pull/2640

NIFI-4456: Support multiple JSON objects in JSON record reader/writers

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [x] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mattyb149/nifi NIFI-4456

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2640.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2640


commit 16ca567ea25e931679ca6f81ff020a94ab7be32d
Author: Matthew Burgess 
Date:   2018-04-17T15:04:56Z

NIFI-4456: Support multiple JSON objects in JSON record reader/writers




---

[jira] [Commented] (NIFIREG-163) Add JDK requirement to Admin Guide

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFIREG-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441249#comment-16441249
 ] 

ASF GitHub Bot commented on NIFIREG-163:


Github user scottyaslan commented on the issue:

https://github.com/apache/nifi-registry/pull/113
  
Reviewing...


> Add JDK requirement to Admin Guide
> --
>
> Key: NIFIREG-163
> URL: https://issues.apache.org/jira/browse/NIFIREG-163
> Project: NiFi Registry
>  Issue Type: Improvement
>Affects Versions: 0.1.0
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Minor
>
> Should notify users of the JDK requirement and potential error that results 
> as discussed in NIFIREG-142



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-registry issue #113: NIFIREG-163 Added JDK information to System Requir...

[scottyaslan]

Github user scottyaslan commented on the issue:

https://github.com/apache/nifi-registry/pull/113
  
Reviewing...


---

[jira] [Commented] (NIFIREG-163) Add JDK requirement to Admin Guide

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFIREG-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441245#comment-16441245
 ] 

ASF GitHub Bot commented on NIFIREG-163:


GitHub user andrewmlim opened a pull request:

https://github.com/apache/nifi-registry/pull/113

NIFIREG-163 Added JDK information to System Requirements section of A…

…dmin Guide

Also added formatting to a default system property to be consistent with 
others.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/andrewmlim/nifi-registry NIFIREG-163

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-registry/pull/113.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #113


commit 64e72cd494d70d07c8ea9a63b7248fa16017dbc0
Author: Andrew Lim 
Date:   2018-04-17T17:44:48Z

NIFIREG-163 Added JDK information to System Requirements section of Admin 
Guide




> Add JDK requirement to Admin Guide
> --
>
> Key: NIFIREG-163
> URL: https://issues.apache.org/jira/browse/NIFIREG-163
> Project: NiFi Registry
>  Issue Type: Improvement
>Affects Versions: 0.1.0
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Minor
>
> Should notify users of the JDK requirement and potential error that results 
> as discussed in NIFIREG-142



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-registry pull request #113: NIFIREG-163 Added JDK information to System...

[andrewmlim]

GitHub user andrewmlim opened a pull request:

https://github.com/apache/nifi-registry/pull/113

NIFIREG-163 Added JDK information to System Requirements section of A…

…dmin Guide

Also added formatting to a default system property to be consistent with 
others.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/andrewmlim/nifi-registry NIFIREG-163

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-registry/pull/113.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #113


commit 64e72cd494d70d07c8ea9a63b7248fa16017dbc0
Author: Andrew Lim 
Date:   2018-04-17T17:44:48Z

NIFIREG-163 Added JDK information to System Requirements section of Admin 
Guide




---

[jira] [Commented] (MINIFICPP-455) Implement escape/unescape HTML4 EL functions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/MINIFICPP-455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441215#comment-16441215
 ] 

ASF GitHub Bot commented on MINIFICPP-455:
--

GitHub user achristianson opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/300

MINIFICPP-455 Added HTML4 escape/unescape

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
 in the commit message?

- [x] Does your PR title start with MINIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] If applicable, have you updated the LICENSE file?
- [x] If applicable, have you updated the NOTICE file?

### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFICPP-455

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/300.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #300


commit 14ebde3170310f64f9cdfa9af7fecad8926fefa4
Author: Andrew I. Christianson 
Date:   2018-04-17T17:20:48Z

MINIFICPP-455 Added HTML4 escape/unescape




> Implement escape/unescape HTML4 EL functions
> 
>
> Key: MINIFICPP-455
> URL: https://issues.apache.org/jira/browse/MINIFICPP-455
> Project: NiFi MiNiFi C++
>  Issue Type: Improvement
>Reporter: Andrew Christianson
>Assignee: Andrew Christianson
>Priority: Major
>
> [Encode/Decode 
> Functions|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#encode]
>  * 
> [escapeHtml4|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#escapehtml4]
>  * 
> [unescapeHtml4|https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#unescapehtml4]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi-minifi-cpp pull request #300: MINIFICPP-455 Added HTML4 escape/unescape

[achristianson]

GitHub user achristianson opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/300

MINIFICPP-455 Added HTML4 escape/unescape

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
 in the commit message?

- [x] Does your PR title start with MINIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] If applicable, have you updated the LICENSE file?
- [x] If applicable, have you updated the NOTICE file?

### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFICPP-455

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/300.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #300


commit 14ebde3170310f64f9cdfa9af7fecad8926fefa4
Author: Andrew I. Christianson 
Date:   2018-04-17T17:20:48Z

MINIFICPP-455 Added HTML4 escape/unescape




---

[jira] [Created] (NIFIREG-163) Add JDK requirement to Admin Guide

[Andrew Lim (JIRA)]

Andrew Lim created NIFIREG-163:
--

 Summary: Add JDK requirement to Admin Guide
 Key: NIFIREG-163
 URL: https://issues.apache.org/jira/browse/NIFIREG-163
 Project: NiFi Registry
  Issue Type: Improvement
Affects Versions: 0.1.0
Reporter: Andrew Lim


Should notify users of the JDK requirement and potential error that results as 
discussed in NIFIREG-142



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (NIFIREG-163) Add JDK requirement to Admin Guide

[Andrew Lim (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFIREG-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Lim reassigned NIFIREG-163:
--

Assignee: Andrew Lim

> Add JDK requirement to Admin Guide
> --
>
> Key: NIFIREG-163
> URL: https://issues.apache.org/jira/browse/NIFIREG-163
> Project: NiFi Registry
>  Issue Type: Improvement
>Affects Versions: 0.1.0
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Minor
>
> Should notify users of the JDK requirement and potential error that results 
> as discussed in NIFIREG-142



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4906) Add GetHdfsFileInfo Processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441080#comment-16441080
 ] 

ASF GitHub Bot commented on NIFI-4906:
--

GitHub user bdesert opened a pull request:

https://github.com/apache/nifi/pull/2639

NIFI-4906 Add GetHDFSFileInfo

NIFI-4906: Added support to scan HDFS to get files and directories 
information without pulling the files.
---

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [x] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bdesert/nifi NIFI-4906-Add-GetHdfsFileInfo

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2639.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2639


commit 8f371524ca9ee09af288e0883e1e3f3861034af9
Author: Ed 
Date:   2018-04-17T14:26:50Z

NIFI-4906 Add GetHDFSFileInfo




> Add GetHdfsFileInfo Processor
> -
>
> Key: NIFI-4906
> URL: https://issues.apache.org/jira/browse/NIFI-4906
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Ed Berezitsky
>Assignee: Ed Berezitsky
>Priority: Major
> Attachments: NiFi-GetHDFSFileInfo.pdf
>
>
> Add *GetHdfsFileInfo* Processor to be able to get stats from a file system.
> This processor should support recursive scan, getting information of 
> directories and files.
> _File-level info required_: name, path, length, modified timestamp, last 
> access timestamp, owner, group, permissions.
> _Directory-level info required_: name, path, sum of lengths of files under a 
> dir, count of files under a dir, modified timestamp, last access timestamp, 
> owner, group, permissions.
>  
> The result returned:
>  * in single flow file (in content - a json line per file/dir info);
>  * flow file per each file/dir info (in content as json obj or in set of 
> attributes by the choice).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2639: NIFI-4906 Add GetHDFSFileInfo

[bdesert]

GitHub user bdesert opened a pull request:

https://github.com/apache/nifi/pull/2639

NIFI-4906 Add GetHDFSFileInfo

NIFI-4906: Added support to scan HDFS to get files and directories 
information without pulling the files.
---

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [x] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bdesert/nifi NIFI-4906-Add-GetHdfsFileInfo

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2639.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2639


commit 8f371524ca9ee09af288e0883e1e3f3861034af9
Author: Ed 
Date:   2018-04-17T14:26:50Z

NIFI-4906 Add GetHDFSFileInfo




---

[jira] [Commented] (NIFI-4516) Add FetchSolr processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441051#comment-16441051
 ] 

ASF GitHub Bot commented on NIFI-4516:
--

Github user JohannesDaniel commented on the issue:

https://github.com/apache/nifi/pull/2517
  
@MikeThomsen 
- added provenance receive notifications
- added tests for attributes
- added upper limit of solr start parameter (1)
- enhanced documentation for upper limit of start param and parameters for 
facet and stats


> Add FetchSolr processor
> ---
>
> Key: NIFI-4516
> URL: https://issues.apache.org/jira/browse/NIFI-4516
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Johannes Peter
>Assignee: Johannes Peter
>Priority: Major
>  Labels: features
>
> The processor shall be capable 
> * to query Solr within a workflow,
> * to make use of standard functionalities of Solr such as faceting, 
> highlighting, result grouping, etc.,
> * to make use of NiFis expression language to build Solr queries, 
> * to handle results as records.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2517: NIFI-4516 FetchSolr Processor

[JohannesDaniel]

Github user JohannesDaniel commented on the issue:

https://github.com/apache/nifi/pull/2517
  
@MikeThomsen 
- added provenance receive notifications
- added tests for attributes
- added upper limit of solr start parameter (1)
- enhanced documentation for upper limit of start param and parameters for 
facet and stats


---

[jira] [Updated] (NIFI-5082) SQL processors do not handle Avro conversion of Oracle timestamps correctly

[Matt Burgess (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess updated NIFI-5082:
---
Status: Patch Available  (was: In Progress)

> SQL processors do not handle Avro conversion of Oracle timestamps correctly
> ---
>
> Key: NIFI-5082
> URL: https://issues.apache.org/jira/browse/NIFI-5082
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
>
> In JdbcCommon (used by such processors as ExecuteSQL and QueryDatabaseTable), 
> if a ResultSet column is not a CLOB or BLOB, its value is retrieved using 
> getObject(), then further processing is done based on the SQL type and/or the 
> Java class of the value.
> However, in Oracle when getObject() is called on a Timestamp column, it 
> returns an Oracle-specific TIMESTAMP class which does not inherit from 
> java.sql.Timestamp or java.sql.Date. Thus the processing "falls through" and 
> its value is attempted to be inserted as a string, which violates the Avro 
> schema (which correctly recognized it as a long of timestamp logical type).
> At least for Oracle, the right way to process a Timestamp column is to call 
> getTimestamp() rather than getObject(), the former returns a 
> java.sql.Timestamp object which would correctly be processed by the current 
> code. I would hope that all drivers would support this but we would want to 
> test on (at least) MySQL, Oracle, and PostgreSQL.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5082) SQL processors do not handle Avro conversion of Oracle timestamps correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440950#comment-16440950
 ] 

ASF GitHub Bot commented on NIFI-5082:
--

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2638#discussion_r182100896
  
--- Diff: 
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/ResultSetRecordSet.java
 ---
@@ -350,6 +350,8 @@ private static RecordFieldType getFieldType(final int 
sqlType) {
 return RecordFieldType.TIME;
 case Types.TIMESTAMP:
 case Types.TIMESTAMP_WITH_TIMEZONE:
+case -101: // Oracle's TIMESTAMP WITH TIME ZONE
--- End diff --

@markap14 I don't think we need these cases (Calcite would never return 
Oracle types, and there'd be no other way to generate them?) but put them in 
just in case (as this does Java SQL Timestamp handling). If you agree then I 
can remove this (although it's likely no-harm-no-foul)


> SQL processors do not handle Avro conversion of Oracle timestamps correctly
> ---
>
> Key: NIFI-5082
> URL: https://issues.apache.org/jira/browse/NIFI-5082
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
>
> In JdbcCommon (used by such processors as ExecuteSQL and QueryDatabaseTable), 
> if a ResultSet column is not a CLOB or BLOB, its value is retrieved using 
> getObject(), then further processing is done based on the SQL type and/or the 
> Java class of the value.
> However, in Oracle when getObject() is called on a Timestamp column, it 
> returns an Oracle-specific TIMESTAMP class which does not inherit from 
> java.sql.Timestamp or java.sql.Date. Thus the processing "falls through" and 
> its value is attempted to be inserted as a string, which violates the Avro 
> schema (which correctly recognized it as a long of timestamp logical type).
> At least for Oracle, the right way to process a Timestamp column is to call 
> getTimestamp() rather than getObject(), the former returns a 
> java.sql.Timestamp object which would correctly be processed by the current 
> code. I would hope that all drivers would support this but we would want to 
> test on (at least) MySQL, Oracle, and PostgreSQL.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2638: NIFI-5082: Added support for custom Oracle timestam...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2638#discussion_r182100896
  
--- Diff: 
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/ResultSetRecordSet.java
 ---
@@ -350,6 +350,8 @@ private static RecordFieldType getFieldType(final int 
sqlType) {
 return RecordFieldType.TIME;
 case Types.TIMESTAMP:
 case Types.TIMESTAMP_WITH_TIMEZONE:
+case -101: // Oracle's TIMESTAMP WITH TIME ZONE
--- End diff --

@markap14 I don't think we need these cases (Calcite would never return 
Oracle types, and there'd be no other way to generate them?) but put them in 
just in case (as this does Java SQL Timestamp handling). If you agree then I 
can remove this (although it's likely no-harm-no-foul)


---

[jira] [Commented] (NIFI-5082) SQL processors do not handle Avro conversion of Oracle timestamps correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440948#comment-16440948
 ] 

ASF GitHub Bot commented on NIFI-5082:
--

Github user mattyb149 commented on the issue:

https://github.com/apache/nifi/pull/2638
  
I REALLY didn't want to have to do this explicit Oracle stuff in the 
generic code, but the other options I tried weren't viable. I tried isolating 
the SQL type conversion to the DatabaseAdapters, but ExecuteSQL uses JdbcCommon 
and does not use DatabaseAdapters, so the problem would still exist. I also 
tried calling getTimestamp() in the default handling block (before attempting 
to store a String) but that is really brittle since other drivers may or may 
not handle getTimestamp() in a manner we could rely on. This is a simple 
straightforward (but dirty) fix.


> SQL processors do not handle Avro conversion of Oracle timestamps correctly
> ---
>
> Key: NIFI-5082
> URL: https://issues.apache.org/jira/browse/NIFI-5082
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
>
> In JdbcCommon (used by such processors as ExecuteSQL and QueryDatabaseTable), 
> if a ResultSet column is not a CLOB or BLOB, its value is retrieved using 
> getObject(), then further processing is done based on the SQL type and/or the 
> Java class of the value.
> However, in Oracle when getObject() is called on a Timestamp column, it 
> returns an Oracle-specific TIMESTAMP class which does not inherit from 
> java.sql.Timestamp or java.sql.Date. Thus the processing "falls through" and 
> its value is attempted to be inserted as a string, which violates the Avro 
> schema (which correctly recognized it as a long of timestamp logical type).
> At least for Oracle, the right way to process a Timestamp column is to call 
> getTimestamp() rather than getObject(), the former returns a 
> java.sql.Timestamp object which would correctly be processed by the current 
> code. I would hope that all drivers would support this but we would want to 
> test on (at least) MySQL, Oracle, and PostgreSQL.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5082) SQL processors do not handle Avro conversion of Oracle timestamps correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440941#comment-16440941
 ] 

ASF GitHub Bot commented on NIFI-5082:
--

GitHub user mattyb149 opened a pull request:

https://github.com/apache/nifi/pull/2638

NIFI-5082: Added support for custom Oracle timestamp types to Avro 
conversion

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mattyb149/nifi NIFI-5082

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2638.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2638


commit 739cccf468a505a8d9866e847fbbb2a37a24817a
Author: Matthew Burgess 
Date:   2018-04-17T14:44:26Z

NIFI-5082: Added support for custom Oracle timestamp types to Avro 
conversion




> SQL processors do not handle Avro conversion of Oracle timestamps correctly
> ---
>
> Key: NIFI-5082
> URL: https://issues.apache.org/jira/browse/NIFI-5082
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
>
> In JdbcCommon (used by such processors as ExecuteSQL and QueryDatabaseTable), 
> if a ResultSet column is not a CLOB or BLOB, its value is retrieved using 
> getObject(), then further processing is done based on the SQL type and/or the 
> Java class of the value.
> However, in Oracle when getObject() is called on a Timestamp column, it 
> returns an Oracle-specific TIMESTAMP class which does not inherit from 
> java.sql.Timestamp or java.sql.Date. Thus the processing "falls through" and 
> its value is attempted to be inserted as a string, which violates the Avro 
> schema (which correctly recognized it as a long of timestamp logical type).
> At least for Oracle, the right way to process a Timestamp column is to call 
> getTimestamp() rather than getObject(), the former returns a 
> java.sql.Timestamp object which would correctly be processed by the current 
> code. I would hope that all drivers would support this but we would want to 
> test on (at least) MySQL, Oracle, and PostgreSQL.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi pull request #2638: NIFI-5082: Added support for custom Oracle timestam...

[mattyb149]

GitHub user mattyb149 opened a pull request:

https://github.com/apache/nifi/pull/2638

NIFI-5082: Added support for custom Oracle timestamp types to Avro 
conversion

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mattyb149/nifi NIFI-5082

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2638.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2638


commit 739cccf468a505a8d9866e847fbbb2a37a24817a
Author: Matthew Burgess 
Date:   2018-04-17T14:44:26Z

NIFI-5082: Added support for custom Oracle timestamp types to Avro 
conversion




---

[jira] [Commented] (NIFI-5066) Support enable and disable component action when multiple components selected or when selecting a process group.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440874#comment-16440874
 ] 

ASF GitHub Bot commented on NIFI-5066:
--

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/2633
  
@andrewmlim When no components are selected both enable/disable and 
start/stop should be allowed as the request will be applied to all eligible 
components in the current process group.


> Support enable and disable component action when multiple components selected 
> or when selecting a process group.
> 
>
> Key: NIFI-5066
> URL: https://issues.apache.org/jira/browse/NIFI-5066
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Affects Versions: 1.5.0
>Reporter: Matthew Clarke
>Assignee: Matt Gilman
>Priority: Major
>
> Currently NiFi validates all processors that are in a STOPPED state.  To 
> reduce impact when flows contain very large numbers of STOPPED processors, 
> users should be disabling these STOPPED processors.  NiFi's "Enable" and 
> "Disable" buttons do not support being used when more then one processor is 
> selected.  When needing to enable or disable large numbers of processors, 
> this is less then ideal. The Enable and Disable buttons should work similar 
> to how the Start and Stop buttons work.
> Have multiple components selected or a process group selected.  Select 
> "Enable" or "Disabled" button.  Any eligible component (those that are not 
> running) should be either enabled or disabled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2633: NIFI-5066: Allow global enable/disable component requests

[mcgilman]

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/2633
  
@andrewmlim When no components are selected both enable/disable and 
start/stop should be allowed as the request will be applied to all eligible 
components in the current process group.


---

[jira] [Commented] (NIFI-5066) Support enable and disable component action when multiple components selected or when selecting a process group.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440864#comment-16440864
 ] 

ASF GitHub Bot commented on NIFI-5066:
--

Github user scottyaslan commented on the issue:

https://github.com/apache/nifi/pull/2633
  
+1 for the UI code.


> Support enable and disable component action when multiple components selected 
> or when selecting a process group.
> 
>
> Key: NIFI-5066
> URL: https://issues.apache.org/jira/browse/NIFI-5066
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Affects Versions: 1.5.0
>Reporter: Matthew Clarke
>Assignee: Matt Gilman
>Priority: Major
>
> Currently NiFi validates all processors that are in a STOPPED state.  To 
> reduce impact when flows contain very large numbers of STOPPED processors, 
> users should be disabling these STOPPED processors.  NiFi's "Enable" and 
> "Disable" buttons do not support being used when more then one processor is 
> selected.  When needing to enable or disable large numbers of processors, 
> this is less then ideal. The Enable and Disable buttons should work similar 
> to how the Start and Stop buttons work.
> Have multiple components selected or a process group selected.  Select 
> "Enable" or "Disabled" button.  Any eligible component (those that are not 
> running) should be either enabled or disabled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2633: NIFI-5066: Allow global enable/disable component requests

[scottyaslan]

Github user scottyaslan commented on the issue:

https://github.com/apache/nifi/pull/2633
  
+1 for the UI code.


---

[jira] [Created] (NIFI-5088) Add property for directory ownership/permissions to PutFile

[Mark Bean (JIRA)]

Mark Bean created NIFI-5088:
---

 Summary: Add property for directory ownership/permissions to 
PutFile
 Key: NIFI-5088
 URL: https://issues.apache.org/jira/browse/NIFI-5088
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Extensions
Affects Versions: 1.6.0
Reporter: Mark Bean


The PutFile processor allows setting owner and permissions of files being put. 
A similar setting should be available for the directories the processor 
creates. Suggest the following properties be added and applied to any 
directories created by the processor

Directory Permissions
Directory Owner
Directory Group



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-5081) Lack of guidance and inability to deal with ISO-8601 dates

[Matt Forrester (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Forrester updated NIFI-5081:
-
Description: 
I've got a Node process that outputs in JSON onto an SQS queue. The dates it 
spits out are ISO-8601 dates within a string, which is the normal, default and 
best way to do this in JSON.

I tried putting them into MongoDB with PutMongo and they go in as strings, 
which is not good ( https://issues.apache.org/jira/browse/NIFI-2079 ).

Gave up on Mongo and tried PostgreSQL...

Figuring I was in Java land I used an esoteric path of GetSQS > 
EvaluateJsonPath > UpdateAttribute [ 
"$\{time:toDate("-MM-dd'T'HH:mm:ss.SSS'Z'", "GMT") ] > PutSQL to get it 
into what I assume is a java.lang.Date, it took me forever to find the 
sql.args.N.type's required but for some reason PutSQL does not like 
java.util.Dates.

Eventually found the ConvertJSONToSQL processor and this created my SQL for me, 
but it doesn't work as it leaves ISO-8601 dates as ISO-8601 dates, which don't 
seem to work.

Eventually found this 
[https://community.hortonworks.com/questions/84772/putsql-with-date-as-argument.html]
 and now I have something working, but I'm using my esoteric GetSQS -> 
EvaluateJsonPath -> UpdateAttribute -> PutSQL path again.

Think there should be some documentation around this at least because it's very 
non-obvious.

  was:
I've got a Node process that outputs in JSON onto an SQS queue. The dates it 
spits out are ISO-8601 dates within a string, which is the normal, default and 
best way to do this in JSON.

I tried putting them into MongoDB with PutMongo and they go in as strings, 
which is not good ( https://issues.apache.org/jira/browse/NIFI-2079 ).

Gave up on Mongo and tried PostgreSQL...

Figuring I was in Java land I used an esoteric path of GetSQS > 
EvaluateJsonPath > UpdateAttribute [ 
"$\{time:toDate("-MM-dd'T'HH:mm:ss.SSS'Z'", "GMT") ] > PutSQL to get it 
into what I assume is a java.lang.Date, it took me forever to find the 
sql.args.N.type's required but for some reason PutSQL does not like 
java.util.Dates.

Eventually found the ConvertJSONToSQL processor and this created my SQL for me, 
but it doesn't work as it leaves ISO-8601 dates as ISO-8601 dates, which don't 
seem to work.

Eventually found this 
[https://community.hortonworks.com/questions/84772/putsql-with-date-as-argument.html]
 and now I have something working, but I'm using my esoteric GetSQS -> 
EvaluateJsonPath -> UpdateAttribute -> PutSQL path again.

Having to convert an ISO-8601 date to a number, particularly using the format 
above is pretty terrible. Putting dates in a database should be trivial and 
probably just work with ISO-8601 / java.util.Date. Think there should be some 
documentation around this at least because it's very non-obvious.


> Lack of guidance and inability to deal with ISO-8601 dates
> --
>
> Key: NIFI-5081
> URL: https://issues.apache.org/jira/browse/NIFI-5081
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation  Website
>Affects Versions: 1.6.0
> Environment: Ubuntu / Chromeium
>Reporter: Matt Forrester
>Priority: Minor
> Attachments: y.xml
>
>
> I've got a Node process that outputs in JSON onto an SQS queue. The dates it 
> spits out are ISO-8601 dates within a string, which is the normal, default 
> and best way to do this in JSON.
> I tried putting them into MongoDB with PutMongo and they go in as strings, 
> which is not good ( https://issues.apache.org/jira/browse/NIFI-2079 ).
> Gave up on Mongo and tried PostgreSQL...
> Figuring I was in Java land I used an esoteric path of GetSQS > 
> EvaluateJsonPath > UpdateAttribute [ 
> "$\{time:toDate("-MM-dd'T'HH:mm:ss.SSS'Z'", "GMT") ] > PutSQL to get it 
> into what I assume is a java.lang.Date, it took me forever to find the 
> sql.args.N.type's required but for some reason PutSQL does not like 
> java.util.Dates.
> Eventually found the ConvertJSONToSQL processor and this created my SQL for 
> me, but it doesn't work as it leaves ISO-8601 dates as ISO-8601 dates, which 
> don't seem to work.
> Eventually found this 
> [https://community.hortonworks.com/questions/84772/putsql-with-date-as-argument.html]
>  and now I have something working, but I'm using my esoteric GetSQS -> 
> EvaluateJsonPath -> UpdateAttribute -> PutSQL path again.
> Think there should be some documentation around this at least because it's 
> very non-obvious.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4516) Add FetchSolr processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440791#comment-16440791
 ] 

ASF GitHub Bot commented on NIFI-4516:
--

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2517
  
@JohannesDaniel Sounds good. Make sure it includes the request 
documentation as well. That's going to be a big deal in making this easy for 
new users to use. Thanks.


> Add FetchSolr processor
> ---
>
> Key: NIFI-4516
> URL: https://issues.apache.org/jira/browse/NIFI-4516
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Johannes Peter
>Assignee: Johannes Peter
>Priority: Major
>  Labels: features
>
> The processor shall be capable 
> * to query Solr within a workflow,
> * to make use of standard functionalities of Solr such as faceting, 
> highlighting, result grouping, etc.,
> * to make use of NiFis expression language to build Solr queries, 
> * to handle results as records.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2517: NIFI-4516 FetchSolr Processor

[MikeThomsen]

Github user MikeThomsen commented on the issue:

https://github.com/apache/nifi/pull/2517
  
@JohannesDaniel Sounds good. Make sure it includes the request 
documentation as well. That's going to be a big deal in making this easy for 
new users to use. Thanks.


---

[jira] [Commented] (NIFI-4516) Add FetchSolr processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440779#comment-16440779
 ] 

ASF GitHub Bot commented on NIFI-4516:
--

Github user JohannesDaniel commented on the issue:

https://github.com/apache/nifi/pull/2517
  
(means in 5-6 hous)


> Add FetchSolr processor
> ---
>
> Key: NIFI-4516
> URL: https://issues.apache.org/jira/browse/NIFI-4516
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Johannes Peter
>Assignee: Johannes Peter
>Priority: Major
>  Labels: features
>
> The processor shall be capable 
> * to query Solr within a workflow,
> * to make use of standard functionalities of Solr such as faceting, 
> highlighting, result grouping, etc.,
> * to make use of NiFis expression language to build Solr queries, 
> * to handle results as records.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-4516) Add FetchSolr processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440778#comment-16440778
 ] 

ASF GitHub Bot commented on NIFI-4516:
--

Github user JohannesDaniel commented on the issue:

https://github.com/apache/nifi/pull/2517
  
hi @MikeThomsen I will upload the changes this evening.


> Add FetchSolr processor
> ---
>
> Key: NIFI-4516
> URL: https://issues.apache.org/jira/browse/NIFI-4516
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Johannes Peter
>Assignee: Johannes Peter
>Priority: Major
>  Labels: features
>
> The processor shall be capable 
> * to query Solr within a workflow,
> * to make use of standard functionalities of Solr such as faceting, 
> highlighting, result grouping, etc.,
> * to make use of NiFis expression language to build Solr queries, 
> * to handle results as records.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] nifi issue #2517: NIFI-4516 FetchSolr Processor

[JohannesDaniel]

Github user JohannesDaniel commented on the issue:

https://github.com/apache/nifi/pull/2517
  
(means in 5-6 hous)


---

[GitHub] nifi issue #2517: NIFI-4516 FetchSolr Processor

[JohannesDaniel]

Github user JohannesDaniel commented on the issue:

https://github.com/apache/nifi/pull/2517
  
hi @MikeThomsen I will upload the changes this evening.


---

[jira] [Updated] (NIFI-5081) Lack of guidance and inability to deal with ISO-8601 dates

[Matt Forrester (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Forrester updated NIFI-5081:
-
Description: 
I've got a Node process that outputs in JSON onto an SQS queue. The dates it 
spits out are ISO-8601 dates within a string, which is the normal, default and 
best way to do this in JSON.

I tried putting them into MongoDB with PutMongo and they go in as strings, 
which is not good ( https://issues.apache.org/jira/browse/NIFI-2079 ).

Gave up on Mongo and tried PostgreSQL...

Figuring I was in Java land I used an esoteric path of GetSQS > 
EvaluateJsonPath > UpdateAttribute [ 
"$\{time:toDate("-MM-dd'T'HH:mm:ss.SSS'Z'", "GMT") ] > PutSQL to get it 
into what I assume is a java.lang.Date, it took me forever to find the 
sql.args.N.type's required but for some reason PutSQL does not like 
java.util.Dates.

Eventually found the ConvertJSONToSQL processor and this created my SQL for me, 
but it doesn't work as it leaves ISO-8601 dates as ISO-8601 dates, which don't 
seem to work.

Eventually found this 
[https://community.hortonworks.com/questions/84772/putsql-with-date-as-argument.html]
 and now I have something working, but I'm using my esoteric GetSQS -> 
EvaluateJsonPath -> UpdateAttribute -> PutSQL path again.

Having to convert an ISO-8601 date to a number, particularly using the format 
above is pretty terrible. Putting dates in a database should be trivial and 
probably just work with ISO-8601 / java.util.Date. Think there should be some 
documentation around this at least because it's very non-obvious.

  was:
I've got a Node process that outputs in JSON onto an SQS queue. The dates it 
spits out are ISO-8601 dates within a string, which is the normal, default and 
best way to do this in JSON.

I tried putting them into MongoDB with PutMongo and they go in as strings, 
which is not good ( https://issues.apache.org/jira/browse/NIFI-2079 ).

Gave up on Mongo and tried PostgreSQL...

Figuring I was in Java land I used an esoteric path of GetSQS > 
EvaluateJsonPath > UpdateAttribute [ 
"$\{time:toDate("-MM-dd'T'HH:mm:ss.SSS'Z'", "GMT") ] > PutSQL to get it 
into what I assume is a java.lang.Date, it took me forever to find the 
sql.args.N.type's required but for some reason PutSQL does not like 
java.util.Dates.

Eventually found the ConvertJSONToSQL processor and this created my SQL for me, 
but it doesn't work as it leaves ISO-8601 dates as ISO-8601 dates, which don't 
seem to work.

Eventually found this 
[https://community.hortonworks.com/questions/84772/putsql-with-date-as-argument.html]
 and now I have something working, but I'm using my esoteric GetSQS -> 
EvaluateJsonPath -> UpdateAttribute -> PutSQL path again.

Having to convert an ISO-8601 date to a number, particularly using the format 
above is pretty terrible. Putting dates in a database should be trivial and 
probably just work with ISO-8601 / java.util.Date. Think there should be some 
documentation around this.


> Lack of guidance and inability to deal with ISO-8601 dates
> --
>
> Key: NIFI-5081
> URL: https://issues.apache.org/jira/browse/NIFI-5081
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation  Website
>Affects Versions: 1.6.0
> Environment: Ubuntu / Chromeium
>Reporter: Matt Forrester
>Priority: Minor
> Attachments: y.xml
>
>
> I've got a Node process that outputs in JSON onto an SQS queue. The dates it 
> spits out are ISO-8601 dates within a string, which is the normal, default 
> and best way to do this in JSON.
> I tried putting them into MongoDB with PutMongo and they go in as strings, 
> which is not good ( https://issues.apache.org/jira/browse/NIFI-2079 ).
> Gave up on Mongo and tried PostgreSQL...
> Figuring I was in Java land I used an esoteric path of GetSQS > 
> EvaluateJsonPath > UpdateAttribute [ 
> "$\{time:toDate("-MM-dd'T'HH:mm:ss.SSS'Z'", "GMT") ] > PutSQL to get it 
> into what I assume is a java.lang.Date, it took me forever to find the 
> sql.args.N.type's required but for some reason PutSQL does not like 
> java.util.Dates.
> Eventually found the ConvertJSONToSQL processor and this created my SQL for 
> me, but it doesn't work as it leaves ISO-8601 dates as ISO-8601 dates, which 
> don't seem to work.
> Eventually found this 
> [https://community.hortonworks.com/questions/84772/putsql-with-date-as-argument.html]
>  and now I have something working, but I'm using my esoteric GetSQS -> 
> EvaluateJsonPath -> UpdateAttribute -> PutSQL path again.
> Having to convert an ISO-8601 date to a number, particularly using the format 
> above is pretty terrible. Putting dates in a database should be trivial and 
> probably just work with ISO-8601 / java.util.Date. Think there should be some 
> documentation around 

[jira] [Created] (NIFI-5087) ConvertJSONToSQL creates invalid SQL for PostgreSQL dates

[Matt Forrester (JIRA)]

Matt Forrester created NIFI-5087:


 Summary: ConvertJSONToSQL creates invalid SQL for PostgreSQL dates
 Key: NIFI-5087
 URL: https://issues.apache.org/jira/browse/NIFI-5087
 Project: Apache NiFi
  Issue Type: Bug
Affects Versions: 1.6.0
 Environment: Ubuntu
Reporter: Matt Forrester


Given an ISO-8601 date and a PG Timestamp field, the ConvertJSONToSQL processor 
creates SQL which looks correct but does not work.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-2079) Cannot insert ISO dates correctly

[Matt Forrester (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440669#comment-16440669
 ] 

Matt Forrester commented on NIFI-2079:
--

Converting an ISO-8601 date to a number is not that trivial and is not 
documented except in multiple bugs (because it's not at all obvious that you 
may need to).

> Cannot insert ISO dates correctly
> -
>
> Key: NIFI-2079
> URL: https://issues.apache.org/jira/browse/NIFI-2079
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 0.6.1
>Reporter: asanka sanjaya
>Priority: Major
>
> When try to insert an ISO date to a mongo collection, it always saved as a 
> String instead of ISODate object.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (NIFI-5081) Lack of guidance and inability to deal with ISO-8601 dates

[Matt Forrester (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-5081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Forrester updated NIFI-5081:
-
Description: 
I've got a Node process that outputs in JSON onto an SQS queue. The dates it 
spits out are ISO-8601 dates within a string, which is the normal, default and 
best way to do this in JSON.

I tried putting them into MongoDB with PutMongo and they go in as strings, 
which is not good ( https://issues.apache.org/jira/browse/NIFI-2079 ).

Gave up on Mongo and tried PostgreSQL...

Figuring I was in Java land I used an esoteric path of GetSQS > 
EvaluateJsonPath > UpdateAttribute [ 
"$\{time:toDate("-MM-dd'T'HH:mm:ss.SSS'Z'", "GMT") ] > PutSQL to get it 
into what I assume is a java.lang.Date, it took me forever to find the 
sql.args.N.type's required but for some reason PutSQL does not like 
java.util.Dates.

Eventually found the ConvertJSONToSQL processor and this created my SQL for me, 
but it doesn't work as it leaves ISO-8601 dates as ISO-8601 dates, which don't 
seem to work.

Eventually found this 
[https://community.hortonworks.com/questions/84772/putsql-with-date-as-argument.html]
 and now I have something working, but I'm using my esoteric GetSQS -> 
EvaluateJsonPath -> UpdateAttribute -> PutSQL path again.

Having to convert an ISO-8601 date to a number, particularly using the format 
above is pretty terrible. Putting dates in a database should be trivial and 
probably just work with ISO-8601 / java.util.Date. Think there should be some 
documentation around this.

  was:
I want to apologise for this, it probably seems very rant like and I've tried 
quite hard to make it not sound that way but I think there's some bugs in there 
and probably the need for some documentation too. I figure it is best to give 
you the problem rather than me try to prescribe a solution:

I've got a Node process that outputs in JSON onto an SQS queue. The dates it 
spits out are ISO-8601 dates within a string, which is the normal, default and 
best way to do this in JSON.

I tried putting them into MongoDB with PutMongo and they go in as strings, 
which is not good.

I then learnt all about AVRO and it's logicalTypes because I thought that would 
give enough metadata for PutMongoRecord to know that my ISO-8601 string is 
actually a date. I seem to remember them still going, but still as strings. 
This was now so long ago I cannot be sure...

Gave up on Mongo and tried PostgreSQL...

Figuring I was in Java land I used an esoteric path of GetSQS > 
EvaluateJsonPath > UpdateAttribute [ 
"$\{time:toDate("-MM-dd'T'HH:mm:ss.SSS'Z'", "GMT") ] > PutSQL to get it 
into what I assume is a java.lang.Date, it took me forever to find the 
sql.args.N.type's required but for some reason PutSQL does not like 
java.util.Dates.

Tried formatting them as strings in DD-MM- HH:MM:SS and then putting then 
keeping it in ISO-8601 format as that is also valid in PostgreSQL but no luck.

Eventually found the ConvertJSONToSQL processor and this created my SQL for me, 
but it doesn't work as it leaves ISO-8601 dates as ISO-8601 dates, which don't 
seem to work.

Eventually found this 
[https://community.hortonworks.com/questions/84772/putsql-with-date-as-argument.html]
 and now I have something working, but I'm using my esoteric GetSQS -> 
EvaluateJsonPath -> UpdateAttribute -> PutSQL path again.

In any case it's still rubbish because my format 
"${time:toDate("-MM-dd'T'HH:mm:ss.SSS'Z'", "GMT"):toNumber()}" within 
UpdateAttribute includes a 'Z' to denote UTC but who is to say it's not in the 
"+00:00" style, or indeed from a different time zone.

Surely the only good/correct way to deal with this is to properly parse an 
ISO-8601 date and surely ConvertJSONToSQL should pretty much just work?


> Lack of guidance and inability to deal with ISO-8601 dates
> --
>
> Key: NIFI-5081
> URL: https://issues.apache.org/jira/browse/NIFI-5081
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation  Website
>Affects Versions: 1.6.0
> Environment: Ubuntu / Chromeium
>Reporter: Matt Forrester
>Priority: Minor
> Attachments: y.xml
>
>
> I've got a Node process that outputs in JSON onto an SQS queue. The dates it 
> spits out are ISO-8601 dates within a string, which is the normal, default 
> and best way to do this in JSON.
> I tried putting them into MongoDB with PutMongo and they go in as strings, 
> which is not good ( https://issues.apache.org/jira/browse/NIFI-2079 ).
> Gave up on Mongo and tried PostgreSQL...
> Figuring I was in Java land I used an esoteric path of GetSQS > 
> EvaluateJsonPath > UpdateAttribute [ 
> "$\{time:toDate("-MM-dd'T'HH:mm:ss.SSS'Z'", "GMT") ] > PutSQL to get it 
> into what I assume is a java.lang.Date, it took me forever to 

[jira] [Commented] (NIFI-4942) NiFi Toolkit - Allow migration of master key without previous password

[Andy LoPresto (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440608#comment-16440608
 ] 

Andy LoPresto commented on NIFI-4942:
-

The NiFi Toolkit side has been merged to master by [~YolandaMDavis] but I 
believe there is still some reconciliation needed between us to ensure the 
Ambari side works correctly. 

> NiFi Toolkit - Allow migration of master key without previous password
> --
>
> Key: NIFI-4942
> URL: https://issues.apache.org/jira/browse/NIFI-4942
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Tools and Build
>Affects Versions: 1.5.0
>Reporter: Yolanda M. Davis
>Assignee: Andy LoPresto
>Priority: Major
>
> Currently the encryption cli in nifi toolkit requires that, in order to 
> migrate from one master key to the next, the previous master key or password 
> should be provided. In cases where the provisioning tool doesn't have the 
> previous value available this becomes challenging to provide and may be prone 
> to error. In speaking with [~alopresto] we can allow toolkit to support a 
> mode of execution such that the master key can be updated without requiring 
> the previous password. Also documentation around it's usage should be updated 
> to be clear in describing the purpose and the type of environment where this 
> command should be used (admin only access etc).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)