[jira] [Commented] (NIFI-4892) ValidateCSV Processor - no doublequote escaping in "invalid" output
[ https://issues.apache.org/jira/browse/NIFI-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370662#comment-16370662 ] ASF GitHub Bot commented on NIFI-4892: -- Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/2481 @e-kolpakov fair points. I added a commit to address your comments and check how we process header and valid rows when we have escaped fields. This change means that we will always keep escaping characters in the output (that's why I had to change a unit test). > ValidateCSV Processor - no doublequote escaping in "invalid" output > --- > > Key: NIFI-4892 > URL: https://issues.apache.org/jira/browse/NIFI-4892 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.4.0, 1.5.0 >Reporter: Eugeny Kolpakov >Assignee: Pierre Villard >Priority: Major > Attachments: ValidateCSV_LineByLine_EscapingBug.xml > > > ValidateCSV processor fails to preserve field escaping for flow files routed > to "invalid" relation when "Line by Line" validation strategy is used. > > Minimal sample flow attached. > > Observed in 1.4.0, but 1.5.0 might be affected as well - I'll update this > ticket after upgrading to 1.5.0 (unless someone could do that earlier) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2481: NIFI-4892 - ValidateCSV: no doublequote escaping in invali...
Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/2481 @e-kolpakov fair points. I added a commit to address your comments and check how we process header and valid rows when we have escaped fields. This change means that we will always keep escaping characters in the output (that's why I had to change a unit test). ---
[jira] [Commented] (NIFI-4367) InvokedScriptedProcessor does not support scripted processor that extends AbstractProcessor
[ https://issues.apache.org/jira/browse/NIFI-4367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370657#comment-16370657 ] Patrice Freydiere commented on NIFI-4367: - Hi sorry for the lake of response, i was very busy thoses last days, and 'll have the time to take consideration on this fix and quitely have a deep understanding on it. [~mattyb149] : yes you're right, it not a complete full processor implementation, but it is a lean path for full fledge production proof processor implementation. on our side in France IT, nifi gain into consideration, and scripting capabilities really ease the process into a production proof dataflow. Thanks for your work it really help. > InvokedScriptedProcessor does not support scripted processor that extends > AbstractProcessor > --- > > Key: NIFI-4367 > URL: https://issues.apache.org/jira/browse/NIFI-4367 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Patrice Freydiere >Priority: Major > Labels: InvokeScriptedProcessor, validation > > InvokeScriptedProcessor pass his ValidationContext to the inner script, > validate call > InvokeScriptedProcessor Line 465 :final > Collection instanceResults = instance.validate(context); > > The problem is that the invokedscript pass the ScriptFile PropertyDescriptor > that is validated, if the script derived from the > AbstractConfigurableComponent, (because the AbstractConfigurableComponent > validate all the context properties). > The context should be refined to remove the InvokeScriptedProcessor > Properties. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-3502) Upgrade D3 to the latest 4.x version
[ https://issues.apache.org/jira/browse/NIFI-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370622#comment-16370622 ] ASF GitHub Bot commented on NIFI-3502: -- Github user scottyaslan commented on the issue: https://github.com/apache/nifi/pull/2471 Thanks @mcgilman this has been merged to master. > Upgrade D3 to the latest 4.x version > > > Key: NIFI-3502 > URL: https://issues.apache.org/jira/browse/NIFI-3502 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Scott Aslan >Assignee: Matt Gilman >Priority: Major > > The NIFI canvas web application is using version 3.x of the D3 library which > is a major version behind and is due to be upgraded. This will be a bit of an > effort as the API's have changed in 4.x. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2471: NIFI-3502: Upgrading D3 version
Github user scottyaslan commented on the issue: https://github.com/apache/nifi/pull/2471 Thanks @mcgilman this has been merged to master. ---
[jira] [Commented] (NIFI-3502) Upgrade D3 to the latest 4.x version
[ https://issues.apache.org/jira/browse/NIFI-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370620#comment-16370620 ] ASF GitHub Bot commented on NIFI-3502: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2471 > Upgrade D3 to the latest 4.x version > > > Key: NIFI-3502 > URL: https://issues.apache.org/jira/browse/NIFI-3502 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Scott Aslan >Assignee: Matt Gilman >Priority: Major > > The NIFI canvas web application is using version 3.x of the D3 library which > is a major version behind and is due to be upgraded. This will be a bit of an > effort as the API's have changed in 4.x. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2471: NIFI-3502: Upgrading D3 version
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2471 ---
[jira] [Updated] (NIFI-4857) Record components do not support String <-> byte[] conversions
[ https://issues.apache.org/jira/browse/NIFI-4857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-4857: --- Status: Patch Available (was: In Progress) > Record components do not support String <-> byte[] conversions > -- > > Key: NIFI-4857 > URL: https://issues.apache.org/jira/browse/NIFI-4857 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > > When trying to perform a conversion of a field between a String and a byte > array, various errors are reporting (depending on where the conversion is > taking place). Here are some examples: > 1) CSVReader, if a column with String values is specified in the schema as > "bytes" > 2) ConvertRecord, if an input field is of type String and the output field is > of type "bytes" > 3) ConvertRecord, if an input field is of type "bytes" and the output field > is of type "String" > Many/most/all of the record components use utility methods to convert values, > I believe these methods need to be updated to support conversion between > String and byte[] values. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4857) Record components do not support String <-> byte[] conversions
[ https://issues.apache.org/jira/browse/NIFI-4857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370605#comment-16370605 ] ASF GitHub Bot commented on NIFI-4857: -- GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/2484 NIFI-4857: Added support for String<->byte[] conversions in record readers/writers Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [x] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-4857 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2484.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2484 commit d77e7cc2fe4913ae60ecc34a6fb6da95cb53f766 Author: Matthew Burgess Date: 2018-02-20T20:57:52Z NIFI-4857: Added support for String<->byte[] conversions in record readers/writers > Record components do not support String <-> byte[] conversions > -- > > Key: NIFI-4857 > URL: https://issues.apache.org/jira/browse/NIFI-4857 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > > When trying to perform a conversion of a field between a String and a byte > array, various errors are reporting (depending on where the conversion is > taking place). Here are some examples: > 1) CSVReader, if a column with String values is specified in the schema as > "bytes" > 2) ConvertRecord, if an input field is of type String and the output field is > of type "bytes" > 3) ConvertRecord, if an input field is of type "bytes" and the output field > is of type "String" > Many/most/all of the record components use utility methods to convert values, > I believe these methods need to be updated to support conversion between > String and byte[] values. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2484: NIFI-4857: Added support for String<->byte[] conver...
GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/2484 NIFI-4857: Added support for String<->byte[] conversions in record readers/writers Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [x] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-4857 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2484.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2484 commit d77e7cc2fe4913ae60ecc34a6fb6da95cb53f766 Author: Matthew Burgess Date: 2018-02-20T20:57:52Z NIFI-4857: Added support for String<->byte[] conversions in record readers/writers ---
[jira] [Commented] (NIFI-4872) NIFI component high resource usage annotation
[ https://issues.apache.org/jira/browse/NIFI-4872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370566#comment-16370566 ] ASF GitHub Bot commented on NIFI-4872: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/2475 @jtstorck i'm supportive of it as is. Not in a position right now to test/build. Can try later/tomorrow if someone else hasn't. noticed the travis-ci issues but they appear unrelated > NIFI component high resource usage annotation > - > > Key: NIFI-4872 > URL: https://issues.apache.org/jira/browse/NIFI-4872 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework, Core UI >Affects Versions: 1.5.0 >Reporter: Jeff Storck >Assignee: Jeff Storck >Priority: Critical > > NiFi Processors currently have no means to relay whether or not they have may > be resource intensive or not. The idea here would be to introduce an > Annotation that can be added to Processors that indicate they may cause high > memory, disk, CPU, or network usage. For instance, any Processor that reads > the FlowFile contents into memory (like many XML Processors for instance) may > cause high memory usage. What ultimately determines if there is high > memory/disk/cpu/network usage will depend on the FlowFiles being processed. > With many of these components in the dataflow, it increases the risk of > OutOfMemoryErrors and performance degradation. > The annotation should support one value from a fixed list of: CPU, Disk, > Memory, Network. It should also allow the developer to provide a custom > description of the scenario that the component would fall under the high > usage category. The annotation should be able to be specified multiple > times, for as many resources as it has the potential to be high usage. > By marking components with this new Annotation, we can update the generated > Processor documentation to include this fact. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2475: NIFI-4872 Added annotation for specifying scenarios in whi...
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/2475 @jtstorck i'm supportive of it as is. Not in a position right now to test/build. Can try later/tomorrow if someone else hasn't. noticed the travis-ci issues but they appear unrelated ---
[jira] [Assigned] (NIFI-4887) EncryptedWriteAheadProvenanceRepositoryTest causes build stability issues
[ https://issues.apache.org/jira/browse/NIFI-4887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto reassigned NIFI-4887: --- Assignee: Andy LoPresto > EncryptedWriteAheadProvenanceRepositoryTest causes build stability issues > - > > Key: NIFI-4887 > URL: https://issues.apache.org/jira/browse/NIFI-4887 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0 > Environment: Apache Maven 3.5.0 > Java version: 1.8.0_141, vendor: Oracle Corporation > Java home: > /Library/Java/JavaVirtualMachines/jdk1.8.0_141.jdk/Contents/Home/jre > Default locale: en_US, platform encoding: UTF-8 > OS name: "mac os x", version: "10.13.3", arch: "x86_64", family: "mac" >Reporter: Joseph Witt >Assignee: Andy LoPresto >Priority: Major > Fix For: 1.6.0 > > > Results : > Tests in error: > > EncryptedWriteAheadProvenanceRepositoryTest.testShouldRegisterAndGetEvent:277 > » FileNotFound > Tests run: 118, Failures: 0, Errors: 1, Skipped: 10 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4872) NIFI component high resource usage annotation
[ https://issues.apache.org/jira/browse/NIFI-4872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370407#comment-16370407 ] ASF GitHub Bot commented on NIFI-4872: -- Github user jtstorck commented on the issue: https://github.com/apache/nifi/pull/2475 I agree, @joewitt. I wanted to get the annotation, its integration, and the components that need the annotation tagged, and sort through any issues or changes to the annotation itself before diving too deeply into writing specific descriptions. The annotation supports a description, but it might be that a wall of text might not be the best way to convey a system resource consideration. It might be a good time to look into supporting some formatting of the content in the annotation's description (including Reads/WritesAttribute). > NIFI component high resource usage annotation > - > > Key: NIFI-4872 > URL: https://issues.apache.org/jira/browse/NIFI-4872 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework, Core UI >Affects Versions: 1.5.0 >Reporter: Jeff Storck >Assignee: Jeff Storck >Priority: Critical > > NiFi Processors currently have no means to relay whether or not they have may > be resource intensive or not. The idea here would be to introduce an > Annotation that can be added to Processors that indicate they may cause high > memory, disk, CPU, or network usage. For instance, any Processor that reads > the FlowFile contents into memory (like many XML Processors for instance) may > cause high memory usage. What ultimately determines if there is high > memory/disk/cpu/network usage will depend on the FlowFiles being processed. > With many of these components in the dataflow, it increases the risk of > OutOfMemoryErrors and performance degradation. > The annotation should support one value from a fixed list of: CPU, Disk, > Memory, Network. It should also allow the developer to provide a custom > description of the scenario that the component would fall under the high > usage category. The annotation should be able to be specified multiple > times, for as many resources as it has the potential to be high usage. > By marking components with this new Annotation, we can update the generated > Processor documentation to include this fact. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2475: NIFI-4872 Added annotation for specifying scenarios in whi...
Github user jtstorck commented on the issue: https://github.com/apache/nifi/pull/2475 I agree, @joewitt. I wanted to get the annotation, its integration, and the components that need the annotation tagged, and sort through any issues or changes to the annotation itself before diving too deeply into writing specific descriptions. The annotation supports a description, but it might be that a wall of text might not be the best way to convey a system resource consideration. It might be a good time to look into supporting some formatting of the content in the annotation's description (including Reads/WritesAttribute). ---
[jira] [Commented] (NIFI-4872) NIFI component high resource usage annotation
[ https://issues.apache.org/jira/browse/NIFI-4872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370393#comment-16370393 ] ASF GitHub Bot commented on NIFI-4872: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/2475 I think my only concern is that as-is we're labeling a bunch of things as "CPU" or "MEMORY" but not giving descriptions. As a user i'd see that and thing 'well, how does this use memory'? For instance, does that mean each flowfile's content is fully loaded in memory? Or does it mean part of one is? Or all of a batch of them? Or if we say CPU usage for compression how should I think about number of threads? Or in the case of compress content it might be worth adding 'MEMORY" and explaining that it is actually really efficient and can handle large objects without ever loading much in memory. So in that case the resource consideration is to alleviate concerns. We're not qualifying the usage consideration as good or bad in this approach. But merely "Hey here is a resource usage consideration you should or might have in mind and here is how this component works in that regard". Does this make sense? So, in that sense I'd like to see us add descriptions to all these things we're tagging. Not saying it is a must for the PR but adding "MEMORY" without explaining might just be alarming > NIFI component high resource usage annotation > - > > Key: NIFI-4872 > URL: https://issues.apache.org/jira/browse/NIFI-4872 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework, Core UI >Affects Versions: 1.5.0 >Reporter: Jeff Storck >Assignee: Jeff Storck >Priority: Critical > > NiFi Processors currently have no means to relay whether or not they have may > be resource intensive or not. The idea here would be to introduce an > Annotation that can be added to Processors that indicate they may cause high > memory, disk, CPU, or network usage. For instance, any Processor that reads > the FlowFile contents into memory (like many XML Processors for instance) may > cause high memory usage. What ultimately determines if there is high > memory/disk/cpu/network usage will depend on the FlowFiles being processed. > With many of these components in the dataflow, it increases the risk of > OutOfMemoryErrors and performance degradation. > The annotation should support one value from a fixed list of: CPU, Disk, > Memory, Network. It should also allow the developer to provide a custom > description of the scenario that the component would fall under the high > usage category. The annotation should be able to be specified multiple > times, for as many resources as it has the potential to be high usage. > By marking components with this new Annotation, we can update the generated > Processor documentation to include this fact. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2475: NIFI-4872 Added annotation for specifying scenarios in whi...
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/2475 I think my only concern is that as-is we're labeling a bunch of things as "CPU" or "MEMORY" but not giving descriptions. As a user i'd see that and thing 'well, how does this use memory'? For instance, does that mean each flowfile's content is fully loaded in memory? Or does it mean part of one is? Or all of a batch of them? Or if we say CPU usage for compression how should I think about number of threads? Or in the case of compress content it might be worth adding 'MEMORY" and explaining that it is actually really efficient and can handle large objects without ever loading much in memory. So in that case the resource consideration is to alleviate concerns. We're not qualifying the usage consideration as good or bad in this approach. But merely "Hey here is a resource usage consideration you should or might have in mind and here is how this component works in that regard". Does this make sense? So, in that sense I'd like to see us add descriptions to all these things w e're tagging. Not saying it is a must for the PR but adding "MEMORY" without explaining might just be alarming ---
[jira] [Commented] (NIFI-4872) NIFI component high resource usage annotation
[
https://issues.apache.org/jira/browse/NIFI-4872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370388#comment-16370388
]
ASF GitHub Bot commented on NIFI-4872:
--
Github user jtstorck commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2475#discussion_r169409286
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/html/HtmlDocumentationWriter.java
---
@@ -727,6 +729,41 @@ protected void writeLink(final XMLStreamWriter
xmlStreamWriter, final String tex
xmlStreamWriter.writeEndElement();
}
+/**
+ * Writes all the system resource considerations for this component
+ *
+ * @param configurableComponent the component to describe
+ * @param xmlStreamWriter the xml stream writer to use
+ * @throws XMLStreamException thrown if there was a problem writing
the XML
+ */
+private void
writeSystemResourceConsiderationInfo(ConfigurableComponent
configurableComponent, XMLStreamWriter xmlStreamWriter)
+throws XMLStreamException {
+
+SystemResourceConsideration[] systemResourceConsiderations =
configurableComponent.getClass().getAnnotationsByType(SystemResourceConsideration.class);
+
+writeSimpleElement(xmlStreamWriter, "h3", "System Resource
Considerations:");
+if (systemResourceConsiderations.length > 0) {
+xmlStreamWriter.writeStartElement("table");
+xmlStreamWriter.writeAttribute("id",
"system-resource-considerations");
+xmlStreamWriter.writeStartElement("tr");
+writeSimpleElement(xmlStreamWriter, "th", "Resource");
+writeSimpleElement(xmlStreamWriter, "th", "Description");
+xmlStreamWriter.writeEndElement();
+for (SystemResourceConsideration systemResourceConsideration :
systemResourceConsiderations) {
+xmlStreamWriter.writeStartElement("tr");
+writeSimpleElement(xmlStreamWriter, "td",
systemResourceConsideration.resource().name());
+// TODO allow for HTML characters here.
--- End diff --
That TODO is also present on the reads/writes attributes code in
HtmlProcessorDocumentationWriter. Since the functionality is similar, I added
the TODO there as well. Will have to talk to @mcgilman about the intention
there.
> NIFI component high resource usage annotation
> -
>
> Key: NIFI-4872
> URL: https://issues.apache.org/jira/browse/NIFI-4872
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Core Framework, Core UI
>Affects Versions: 1.5.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Critical
>
> NiFi Processors currently have no means to relay whether or not they have may
> be resource intensive or not. The idea here would be to introduce an
> Annotation that can be added to Processors that indicate they may cause high
> memory, disk, CPU, or network usage. For instance, any Processor that reads
> the FlowFile contents into memory (like many XML Processors for instance) may
> cause high memory usage. What ultimately determines if there is high
> memory/disk/cpu/network usage will depend on the FlowFiles being processed.
> With many of these components in the dataflow, it increases the risk of
> OutOfMemoryErrors and performance degradation.
> The annotation should support one value from a fixed list of: CPU, Disk,
> Memory, Network. It should also allow the developer to provide a custom
> description of the scenario that the component would fall under the high
> usage category. The annotation should be able to be specified multiple
> times, for as many resources as it has the potential to be high usage.
> By marking components with this new Annotation, we can update the generated
> Processor documentation to include this fact.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2475: NIFI-4872 Added annotation for specifying scenarios...
Github user jtstorck commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2475#discussion_r169409286
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/html/HtmlDocumentationWriter.java
---
@@ -727,6 +729,41 @@ protected void writeLink(final XMLStreamWriter
xmlStreamWriter, final String tex
xmlStreamWriter.writeEndElement();
}
+/**
+ * Writes all the system resource considerations for this component
+ *
+ * @param configurableComponent the component to describe
+ * @param xmlStreamWriter the xml stream writer to use
+ * @throws XMLStreamException thrown if there was a problem writing
the XML
+ */
+private void
writeSystemResourceConsiderationInfo(ConfigurableComponent
configurableComponent, XMLStreamWriter xmlStreamWriter)
+throws XMLStreamException {
+
+SystemResourceConsideration[] systemResourceConsiderations =
configurableComponent.getClass().getAnnotationsByType(SystemResourceConsideration.class);
+
+writeSimpleElement(xmlStreamWriter, "h3", "System Resource
Considerations:");
+if (systemResourceConsiderations.length > 0) {
+xmlStreamWriter.writeStartElement("table");
+xmlStreamWriter.writeAttribute("id",
"system-resource-considerations");
+xmlStreamWriter.writeStartElement("tr");
+writeSimpleElement(xmlStreamWriter, "th", "Resource");
+writeSimpleElement(xmlStreamWriter, "th", "Description");
+xmlStreamWriter.writeEndElement();
+for (SystemResourceConsideration systemResourceConsideration :
systemResourceConsiderations) {
+xmlStreamWriter.writeStartElement("tr");
+writeSimpleElement(xmlStreamWriter, "td",
systemResourceConsideration.resource().name());
+// TODO allow for HTML characters here.
--- End diff --
That TODO is also present on the reads/writes attributes code in
HtmlProcessorDocumentationWriter. Since the functionality is similar, I added
the TODO there as well. Will have to talk to @mcgilman about the intention
there.
---
[jira] [Commented] (NIFI-4872) NIFI component high resource usage annotation
[
https://issues.apache.org/jira/browse/NIFI-4872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370385#comment-16370385
]
ASF GitHub Bot commented on NIFI-4872:
--
Github user joewitt commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2475#discussion_r169408354
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/html/HtmlDocumentationWriter.java
---
@@ -727,6 +729,41 @@ protected void writeLink(final XMLStreamWriter
xmlStreamWriter, final String tex
xmlStreamWriter.writeEndElement();
}
+/**
+ * Writes all the system resource considerations for this component
+ *
+ * @param configurableComponent the component to describe
+ * @param xmlStreamWriter the xml stream writer to use
+ * @throws XMLStreamException thrown if there was a problem writing
the XML
+ */
+private void
writeSystemResourceConsiderationInfo(ConfigurableComponent
configurableComponent, XMLStreamWriter xmlStreamWriter)
+throws XMLStreamException {
+
+SystemResourceConsideration[] systemResourceConsiderations =
configurableComponent.getClass().getAnnotationsByType(SystemResourceConsideration.class);
+
+writeSimpleElement(xmlStreamWriter, "h3", "System Resource
Considerations:");
+if (systemResourceConsiderations.length > 0) {
+xmlStreamWriter.writeStartElement("table");
+xmlStreamWriter.writeAttribute("id",
"system-resource-considerations");
+xmlStreamWriter.writeStartElement("tr");
+writeSimpleElement(xmlStreamWriter, "th", "Resource");
+writeSimpleElement(xmlStreamWriter, "th", "Description");
+xmlStreamWriter.writeEndElement();
+for (SystemResourceConsideration systemResourceConsideration :
systemResourceConsiderations) {
+xmlStreamWriter.writeStartElement("tr");
+writeSimpleElement(xmlStreamWriter, "td",
systemResourceConsideration.resource().name());
+// TODO allow for HTML characters here.
--- End diff --
probably need/want to sort out this todo?
> NIFI component high resource usage annotation
> -
>
> Key: NIFI-4872
> URL: https://issues.apache.org/jira/browse/NIFI-4872
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Core Framework, Core UI
>Affects Versions: 1.5.0
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>Priority: Critical
>
> NiFi Processors currently have no means to relay whether or not they have may
> be resource intensive or not. The idea here would be to introduce an
> Annotation that can be added to Processors that indicate they may cause high
> memory, disk, CPU, or network usage. For instance, any Processor that reads
> the FlowFile contents into memory (like many XML Processors for instance) may
> cause high memory usage. What ultimately determines if there is high
> memory/disk/cpu/network usage will depend on the FlowFiles being processed.
> With many of these components in the dataflow, it increases the risk of
> OutOfMemoryErrors and performance degradation.
> The annotation should support one value from a fixed list of: CPU, Disk,
> Memory, Network. It should also allow the developer to provide a custom
> description of the scenario that the component would fall under the high
> usage category. The annotation should be able to be specified multiple
> times, for as many resources as it has the potential to be high usage.
> By marking components with this new Annotation, we can update the generated
> Processor documentation to include this fact.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2475: NIFI-4872 Added annotation for specifying scenarios...
Github user joewitt commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2475#discussion_r169408354
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/html/HtmlDocumentationWriter.java
---
@@ -727,6 +729,41 @@ protected void writeLink(final XMLStreamWriter
xmlStreamWriter, final String tex
xmlStreamWriter.writeEndElement();
}
+/**
+ * Writes all the system resource considerations for this component
+ *
+ * @param configurableComponent the component to describe
+ * @param xmlStreamWriter the xml stream writer to use
+ * @throws XMLStreamException thrown if there was a problem writing
the XML
+ */
+private void
writeSystemResourceConsiderationInfo(ConfigurableComponent
configurableComponent, XMLStreamWriter xmlStreamWriter)
+throws XMLStreamException {
+
+SystemResourceConsideration[] systemResourceConsiderations =
configurableComponent.getClass().getAnnotationsByType(SystemResourceConsideration.class);
+
+writeSimpleElement(xmlStreamWriter, "h3", "System Resource
Considerations:");
+if (systemResourceConsiderations.length > 0) {
+xmlStreamWriter.writeStartElement("table");
+xmlStreamWriter.writeAttribute("id",
"system-resource-considerations");
+xmlStreamWriter.writeStartElement("tr");
+writeSimpleElement(xmlStreamWriter, "th", "Resource");
+writeSimpleElement(xmlStreamWriter, "th", "Description");
+xmlStreamWriter.writeEndElement();
+for (SystemResourceConsideration systemResourceConsideration :
systemResourceConsiderations) {
+xmlStreamWriter.writeStartElement("tr");
+writeSimpleElement(xmlStreamWriter, "td",
systemResourceConsideration.resource().name());
+// TODO allow for HTML characters here.
--- End diff --
probably need/want to sort out this todo?
---
[jira] [Commented] (NIFI-4872) NIFI component high resource usage annotation
[ https://issues.apache.org/jira/browse/NIFI-4872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370379#comment-16370379 ] ASF GitHub Bot commented on NIFI-4872: -- Github user jtstorck commented on the issue: https://github.com/apache/nifi/pull/2475 @joewitt PR has been rebased against current master, and I've implemented some of the changes you requested. > NIFI component high resource usage annotation > - > > Key: NIFI-4872 > URL: https://issues.apache.org/jira/browse/NIFI-4872 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework, Core UI >Affects Versions: 1.5.0 >Reporter: Jeff Storck >Assignee: Jeff Storck >Priority: Critical > > NiFi Processors currently have no means to relay whether or not they have may > be resource intensive or not. The idea here would be to introduce an > Annotation that can be added to Processors that indicate they may cause high > memory, disk, CPU, or network usage. For instance, any Processor that reads > the FlowFile contents into memory (like many XML Processors for instance) may > cause high memory usage. What ultimately determines if there is high > memory/disk/cpu/network usage will depend on the FlowFiles being processed. > With many of these components in the dataflow, it increases the risk of > OutOfMemoryErrors and performance degradation. > The annotation should support one value from a fixed list of: CPU, Disk, > Memory, Network. It should also allow the developer to provide a custom > description of the scenario that the component would fall under the high > usage category. The annotation should be able to be specified multiple > times, for as many resources as it has the potential to be high usage. > By marking components with this new Annotation, we can update the generated > Processor documentation to include this fact. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2475: NIFI-4872 Added annotation for specifying scenarios in whi...
Github user jtstorck commented on the issue: https://github.com/apache/nifi/pull/2475 @joewitt PR has been rebased against current master, and I've implemented some of the changes you requested. ---
[jira] [Commented] (NIFI-4895) Implement back-end for terminating a Processor
[ https://issues.apache.org/jira/browse/NIFI-4895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370375#comment-16370375 ] ASF GitHub Bot commented on NIFI-4895: -- Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/2483 This PR depends on NIFI-4849 so will submit a new PR once that is merged. > Implement back-end for terminating a Processor > -- > > Key: NIFI-4895 > URL: https://issues.apache.org/jira/browse/NIFI-4895 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2483: NIFI-4895
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/2483 This PR depends on NIFI-4849 so will submit a new PR once that is merged. ---
[GitHub] nifi issue #2180: Added GetMongoAggregation to support running Mongo aggrega...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2180 @mattyb149 Changes are in and it's building. ---
[GitHub] nifi pull request #2180: Added GetMongoAggregation to support running Mongo ...
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2180#discussion_r169402643
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/RunMongoAggregation.java
---
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.nifi.processors.mongodb;
+
+import com.mongodb.BasicDBObject;
+import com.mongodb.client.AggregateIterable;
+import com.mongodb.client.MongoCollection;
+import com.mongodb.client.MongoCursor;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.bson.conversions.Bson;
+import com.fasterxml.jackson.databind.ObjectMapper;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+@Tags({"mongo", "aggregation", "aggregate"})
+@CapabilityDescription("A processor that runs an aggregation query
whenever a flowfile is received.")
+@InputRequirement(InputRequirement.Requirement.INPUT_ALLOWED)
+@EventDriven
+public class RunMongoAggregation extends AbstractMongoProcessor {
+
+private final static Set relationships;
+private final static List propertyDescriptors;
+
+static final Relationship REL_ORIGINAL = new Relationship.Builder()
+.description("The input flowfile gets sent to this
relationship when the query succeeds.")
+.name("original")
+.build();
+static final Relationship REL_FAILURE = new Relationship.Builder()
+.description("The input flowfile gets sent to this
relationship when the query fails.")
+.name("failure")
+.build();
+static final Relationship REL_RESULTS = new Relationship.Builder()
+.description("The result set of the aggregation will be sent
to this relationship.")
+.name("results")
+.build();
+
+static final List buildAggregationQuery(String query) throws
IOException {
+List result = new ArrayList<>();
+
+ObjectMapper mapper = new ObjectMapper();
+List values = mapper.readValue(query, List.class);
+for (Map val : values) {
+result.add(new BasicDBObject(val));
+}
+
+return result;
+}
+
+public static final Validator AGG_VALIDATOR = (subject, value,
context) -> {
+final ValidationResult.Builder builder = new
ValidationResult.Builder();
+builder.subject(subject).input(value);
+
+if (context.isExpressionLanguageSupported(subject) &&
context.isExpressionLanguagePresent(value)) {
+return builder.valid(true).explanation("Contains Expression
Language").build();
+}
+
+String reason = null;
+try {
+buildAggregationQuery(value);
+} catch (final RuntimeException | IOException e) {
+reason = e.getLocalizedMessage();
+}
+
+return builder.explanation(reason).valid(reason == null).build();
+};
+
+static final PropertyDescriptor QUERY = new
PropertyDescriptor.Builder()
+.name("mongo
[GitHub] nifi pull request #2180: Added GetMongoAggregation to support running Mongo ...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2180#discussion_r169402684 --- Diff: nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/resources/docs/org.apache.nifi.processors.mongodb.RunMongoAggregation/additionalDetails.html --- @@ -0,0 +1,44 @@ + + + + + +GetKafka --- End diff -- Done ---
[jira] [Commented] (NIFI-4538) Add Process Group information to Search results
[ https://issues.apache.org/jira/browse/NIFI-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370345#comment-16370345 ] ASF GitHub Bot commented on NIFI-4538: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2364 > Add Process Group information to Search results > --- > > Key: NIFI-4538 > URL: https://issues.apache.org/jira/browse/NIFI-4538 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Matt Burgess >Assignee: Yuri >Priority: Major > Fix For: 1.6.0 > > Attachments: Screenshot from 2017-12-23 21-08-45.png, Screenshot from > 2017-12-23 21-42-24.png > > > When querying for components in the Search bar, no Process Group (PG) > information is displayed. When copies of PGs are made on the canvas, the > search results can be hard to navigate, as you may jump into a different PG > than what you're looking for. > I propose adding (conditionally, based on user permissions) the immediate > parent PG name and/or ID, as well as the top-level PG. In this case I mean > top-level being the highest parent PG except root, unless the component's > immediate parent PG is root, in which case it wouldn't need to be displayed > (or could be displayed as the root PG, albeit a duplicate of the immediate). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4538) Add Process Group information to Search results
[ https://issues.apache.org/jira/browse/NIFI-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370344#comment-16370344 ] ASF GitHub Bot commented on NIFI-4538: -- Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2364 Thanks @yuri1969! This has been merged to master. > Add Process Group information to Search results > --- > > Key: NIFI-4538 > URL: https://issues.apache.org/jira/browse/NIFI-4538 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Matt Burgess >Assignee: Yuri >Priority: Major > Fix For: 1.6.0 > > Attachments: Screenshot from 2017-12-23 21-08-45.png, Screenshot from > 2017-12-23 21-42-24.png > > > When querying for components in the Search bar, no Process Group (PG) > information is displayed. When copies of PGs are made on the canvas, the > search results can be hard to navigate, as you may jump into a different PG > than what you're looking for. > I propose adding (conditionally, based on user permissions) the immediate > parent PG name and/or ID, as well as the top-level PG. In this case I mean > top-level being the highest parent PG except root, unless the component's > immediate parent PG is root, in which case it wouldn't need to be displayed > (or could be displayed as the root PG, albeit a duplicate of the immediate). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (NIFI-4538) Add Process Group information to Search results
[ https://issues.apache.org/jira/browse/NIFI-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman resolved NIFI-4538. --- Resolution: Fixed Fix Version/s: 1.6.0 > Add Process Group information to Search results > --- > > Key: NIFI-4538 > URL: https://issues.apache.org/jira/browse/NIFI-4538 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Matt Burgess >Assignee: Yuri >Priority: Major > Fix For: 1.6.0 > > Attachments: Screenshot from 2017-12-23 21-08-45.png, Screenshot from > 2017-12-23 21-42-24.png > > > When querying for components in the Search bar, no Process Group (PG) > information is displayed. When copies of PGs are made on the canvas, the > search results can be hard to navigate, as you may jump into a different PG > than what you're looking for. > I propose adding (conditionally, based on user permissions) the immediate > parent PG name and/or ID, as well as the top-level PG. In this case I mean > top-level being the highest parent PG except root, unless the component's > immediate parent PG is root, in which case it wouldn't need to be displayed > (or could be displayed as the root PG, albeit a duplicate of the immediate). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2364: NIFI-4538 - Add Process Group information to...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2364 ---
[GitHub] nifi issue #2364: NIFI-4538 - Add Process Group information to...
Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2364 Thanks @yuri1969! This has been merged to master. ---
[jira] [Commented] (NIFI-4895) Implement back-end for terminating a Processor
[ https://issues.apache.org/jira/browse/NIFI-4895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370311#comment-16370311 ] ASF GitHub Bot commented on NIFI-4895: -- GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/2483 NIFI-4895 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-4895 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2483.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2483 commit 5ffc024311c6cc635a189840de5f038cba24b536 Author: Mark Payne Date: 2018-02-02T17:16:36Z NIFI-4849: Implemented REST Endpoint and associated backend code to generate a Diagnostics Report for a Processor commit 06dd683b6b0dab96df48a027618eed71be6f116f Author: Mark Payne Date: 2018-02-16T20:27:30Z NIFI-4895: Added backend code to give users the ability to forcibly terminate a processor if the processor does not complete after clicking Stop > Implement back-end for terminating a Processor > -- > > Key: NIFI-4895 > URL: https://issues.apache.org/jira/browse/NIFI-4895 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4895) Implement back-end for terminating a Processor
[ https://issues.apache.org/jira/browse/NIFI-4895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370312#comment-16370312 ] ASF GitHub Bot commented on NIFI-4895: -- Github user markap14 closed the pull request at: https://github.com/apache/nifi/pull/2483 > Implement back-end for terminating a Processor > -- > > Key: NIFI-4895 > URL: https://issues.apache.org/jira/browse/NIFI-4895 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2483: NIFI-4895
Github user markap14 closed the pull request at: https://github.com/apache/nifi/pull/2483 ---
[GitHub] nifi pull request #2483: NIFI-4895
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/2483 NIFI-4895 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-4895 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2483.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2483 commit 5ffc024311c6cc635a189840de5f038cba24b536 Author: Mark Payne Date: 2018-02-02T17:16:36Z NIFI-4849: Implemented REST Endpoint and associated backend code to generate a Diagnostics Report for a Processor commit 06dd683b6b0dab96df48a027618eed71be6f116f Author: Mark Payne Date: 2018-02-16T20:27:30Z NIFI-4895: Added backend code to give users the ability to forcibly terminate a processor if the processor does not complete after clicking Stop ---
[jira] [Commented] (NIFI-3502) Upgrade D3 to the latest 4.x version
[
https://issues.apache.org/jira/browse/NIFI-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370304#comment-16370304
]
ASF GitHub Bot commented on NIFI-3502:
--
Github user mcgilman commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2471#discussion_r169394276
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-canvas.js
---
@@ -996,34 +1030,42 @@
zoomed = true;
}
-// see if the scale has changed during this
zoom event,
-// we want to only transition when zooming
in/out as running
-// the transitions during pan events is
-var transition = d3.event.sourceEvent.type ===
'wheel' || d3.event.sourceEvent.type === 'mousewheel';
-
// refresh the canvas
refreshed = nfCanvas.View.refresh({
persist: false,
-transition: transition,
+transition:
shouldTransition(d3.event.sourceEvent),
refreshComponents: false,
refreshBirdseye: false
});
})
-.on('zoomend', function () {
-// ensure the canvas was actually refreshed
-if (nfCommon.isDefinedAndNotNull(refreshed)) {
-nfGraph.updateVisibility();
-
-// refresh the birdseye
-refreshed.done(function () {
-nfBirdseye.refresh();
-});
+.on('end', function () {
+if (!isBirdseyeEvent(d3.event.sourceEvent)) {
+// ensure the canvas was actually refreshed
+if
(nfCommon.isDefinedAndNotNull(refreshed)) {
+nfGraph.updateVisibility();
+
+// refresh the birdseye
+refreshed.done(function () {
+nfBirdseye.refresh();
+});
-// persist the users view
-nfCanvasUtils.persistUserView();
+// persist the users view
+nfCanvasUtils.persistUserView();
-// reset the refreshed deferred
-refreshed = null;
+// reset the refreshed deferred
+refreshed = null;
+}
+
+if (panning === false) {
+// deselect as necessary if we are not
panning
+
nfCanvasUtils.getSelection().classed('selected', false);
+
+// update URL deep linking params
+nfCanvasUtils.setURLParameters();
+
+// inform Angular app values have
changed
+nfNgBridge.digest();
--- End diff --
Which part? The .digest() was necessary to address @andrewmlim comments.
> Upgrade D3 to the latest 4.x version
>
>
> Key: NIFI-3502
> URL: https://issues.apache.org/jira/browse/NIFI-3502
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Core UI
>Reporter: Scott Aslan
>Assignee: Matt Gilman
>Priority: Major
>
> The NIFI canvas web application is using version 3.x of the D3 library which
> is a major version behind and is due to be upgraded. This will be a bit of an
> effort as the API's have changed in 4.x.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2471: NIFI-3502: Upgrading D3 version
Github user mcgilman commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2471#discussion_r169394276
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-canvas.js
---
@@ -996,34 +1030,42 @@
zoomed = true;
}
-// see if the scale has changed during this
zoom event,
-// we want to only transition when zooming
in/out as running
-// the transitions during pan events is
-var transition = d3.event.sourceEvent.type ===
'wheel' || d3.event.sourceEvent.type === 'mousewheel';
-
// refresh the canvas
refreshed = nfCanvas.View.refresh({
persist: false,
-transition: transition,
+transition:
shouldTransition(d3.event.sourceEvent),
refreshComponents: false,
refreshBirdseye: false
});
})
-.on('zoomend', function () {
-// ensure the canvas was actually refreshed
-if (nfCommon.isDefinedAndNotNull(refreshed)) {
-nfGraph.updateVisibility();
-
-// refresh the birdseye
-refreshed.done(function () {
-nfBirdseye.refresh();
-});
+.on('end', function () {
+if (!isBirdseyeEvent(d3.event.sourceEvent)) {
+// ensure the canvas was actually refreshed
+if
(nfCommon.isDefinedAndNotNull(refreshed)) {
+nfGraph.updateVisibility();
+
+// refresh the birdseye
+refreshed.done(function () {
+nfBirdseye.refresh();
+});
-// persist the users view
-nfCanvasUtils.persistUserView();
+// persist the users view
+nfCanvasUtils.persistUserView();
-// reset the refreshed deferred
-refreshed = null;
+// reset the refreshed deferred
+refreshed = null;
+}
+
+if (panning === false) {
+// deselect as necessary if we are not
panning
+
nfCanvasUtils.getSelection().classed('selected', false);
+
+// update URL deep linking params
+nfCanvasUtils.setURLParameters();
+
+// inform Angular app values have
changed
+nfNgBridge.digest();
--- End diff --
Which part? The .digest() was necessary to address @andrewmlim comments.
---
[jira] [Commented] (NIFI-3502) Upgrade D3 to the latest 4.x version
[
https://issues.apache.org/jira/browse/NIFI-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370299#comment-16370299
]
ASF GitHub Bot commented on NIFI-3502:
--
Github user scottyaslan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2471#discussion_r169393440
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-canvas.js
---
@@ -996,34 +1030,42 @@
zoomed = true;
}
-// see if the scale has changed during this
zoom event,
-// we want to only transition when zooming
in/out as running
-// the transitions during pan events is
-var transition = d3.event.sourceEvent.type ===
'wheel' || d3.event.sourceEvent.type === 'mousewheel';
-
// refresh the canvas
refreshed = nfCanvas.View.refresh({
persist: false,
-transition: transition,
+transition:
shouldTransition(d3.event.sourceEvent),
refreshComponents: false,
refreshBirdseye: false
});
})
-.on('zoomend', function () {
-// ensure the canvas was actually refreshed
-if (nfCommon.isDefinedAndNotNull(refreshed)) {
-nfGraph.updateVisibility();
-
-// refresh the birdseye
-refreshed.done(function () {
-nfBirdseye.refresh();
-});
+.on('end', function () {
+if (!isBirdseyeEvent(d3.event.sourceEvent)) {
+// ensure the canvas was actually refreshed
+if
(nfCommon.isDefinedAndNotNull(refreshed)) {
+nfGraph.updateVisibility();
+
+// refresh the birdseye
+refreshed.done(function () {
+nfBirdseye.refresh();
+});
-// persist the users view
-nfCanvasUtils.persistUserView();
+// persist the users view
+nfCanvasUtils.persistUserView();
-// reset the refreshed deferred
-refreshed = null;
+// reset the refreshed deferred
+refreshed = null;
+}
+
+if (panning === false) {
+// deselect as necessary if we are not
panning
+
nfCanvasUtils.getSelection().classed('selected', false);
+
+// update URL deep linking params
+nfCanvasUtils.setURLParameters();
+
+// inform Angular app values have
changed
+nfNgBridge.digest();
--- End diff --
This is unnecessary...
> Upgrade D3 to the latest 4.x version
>
>
> Key: NIFI-3502
> URL: https://issues.apache.org/jira/browse/NIFI-3502
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Core UI
>Reporter: Scott Aslan
>Assignee: Matt Gilman
>Priority: Major
>
> The NIFI canvas web application is using version 3.x of the D3 library which
> is a major version behind and is due to be upgraded. This will be a bit of an
> effort as the API's have changed in 4.x.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2471: NIFI-3502: Upgrading D3 version
Github user scottyaslan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2471#discussion_r169393440
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-canvas.js
---
@@ -996,34 +1030,42 @@
zoomed = true;
}
-// see if the scale has changed during this
zoom event,
-// we want to only transition when zooming
in/out as running
-// the transitions during pan events is
-var transition = d3.event.sourceEvent.type ===
'wheel' || d3.event.sourceEvent.type === 'mousewheel';
-
// refresh the canvas
refreshed = nfCanvas.View.refresh({
persist: false,
-transition: transition,
+transition:
shouldTransition(d3.event.sourceEvent),
refreshComponents: false,
refreshBirdseye: false
});
})
-.on('zoomend', function () {
-// ensure the canvas was actually refreshed
-if (nfCommon.isDefinedAndNotNull(refreshed)) {
-nfGraph.updateVisibility();
-
-// refresh the birdseye
-refreshed.done(function () {
-nfBirdseye.refresh();
-});
+.on('end', function () {
+if (!isBirdseyeEvent(d3.event.sourceEvent)) {
+// ensure the canvas was actually refreshed
+if
(nfCommon.isDefinedAndNotNull(refreshed)) {
+nfGraph.updateVisibility();
+
+// refresh the birdseye
+refreshed.done(function () {
+nfBirdseye.refresh();
+});
-// persist the users view
-nfCanvasUtils.persistUserView();
+// persist the users view
+nfCanvasUtils.persistUserView();
-// reset the refreshed deferred
-refreshed = null;
+// reset the refreshed deferred
+refreshed = null;
+}
+
+if (panning === false) {
+// deselect as necessary if we are not
panning
+
nfCanvasUtils.getSelection().classed('selected', false);
+
+// update URL deep linking params
+nfCanvasUtils.setURLParameters();
+
+// inform Angular app values have
changed
+nfNgBridge.digest();
--- End diff --
This is unnecessary...
---
[jira] [Commented] (NIFI-4893) Cannot convert Avro schemas to Record schemas with default value in arrays
[
https://issues.apache.org/jira/browse/NIFI-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370269#comment-16370269
]
ASF GitHub Bot commented on NIFI-4893:
--
Github user gardellajuanpablo commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2480#discussion_r169389143
--- Diff:
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/RecordField.java
---
@@ -76,8 +77,12 @@ public RecordField(final String fieldName, final
DataType dataType, final Object
} else {
this.aliases = Collections.unmodifiableSet(aliases);
}
-
-this.defaultValue = defaultValue;
+if (RecordFieldType.ARRAY.equals(dataType.getFieldType()) &&
(defaultValue instanceof String
+|| defaultValue instanceof Number || defaultValue
instanceof Boolean)) {
--- End diff --
Yes, it solves the reported issue, but it makes the test fails. It is
required to think another solution approach. It is not as simple I initially
thought
> Cannot convert Avro schemas to Record schemas with default value in arrays
> --
>
> Key: NIFI-4893
> URL: https://issues.apache.org/jira/browse/NIFI-4893
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: ALL
>Reporter: Gardella Juan Pablo
>Priority: Major
> Attachments: issue1.zip
>
>
> Given an Avro Schema that has a default array defined, it is not possible to
> be converted to a Nifi Record Schema.
> To reproduce the bug, try to convert the following Avro schema to Record
> Schema:
> {code}
> {
> "type": "record",
> "name": "Foo1",
> "namespace": "foo.namespace",
> "fields": [
> {
> "name": "listOfInt",
> "type": {
> "type": "array",
> "items": "int"
> },
> "doc": "array of ints",
> "default": 0
> }
> ]
> }
> {code}
>
> Using org.apache.nifi.avro.AvroTypeUtil class. Attached a maven project to
> reproduce the issue and also the fix.
> * To reproduce the bug, run "mvn clean test"
> * To test the fix, run "mvn clean test -Ppatch".
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2480: NIFI-4893 Cannot convert Avro schemas to Record sch...
Github user gardellajuanpablo commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2480#discussion_r169389143
--- Diff:
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/RecordField.java
---
@@ -76,8 +77,12 @@ public RecordField(final String fieldName, final
DataType dataType, final Object
} else {
this.aliases = Collections.unmodifiableSet(aliases);
}
-
-this.defaultValue = defaultValue;
+if (RecordFieldType.ARRAY.equals(dataType.getFieldType()) &&
(defaultValue instanceof String
+|| defaultValue instanceof Number || defaultValue
instanceof Boolean)) {
--- End diff --
Yes, it solves the reported issue, but it makes the test fails. It is
required to think another solution approach. It is not as simple I initially
thought
---
[GitHub] nifi pull request #2480: NIFI-4893 Cannot convert Avro schemas to Record sch...
Github user zenfenan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2480#discussion_r169385890
--- Diff:
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/RecordField.java
---
@@ -76,8 +77,12 @@ public RecordField(final String fieldName, final
DataType dataType, final Object
} else {
this.aliases = Collections.unmodifiableSet(aliases);
}
-
-this.defaultValue = defaultValue;
+if (RecordFieldType.ARRAY.equals(dataType.getFieldType()) &&
(defaultValue instanceof String
+|| defaultValue instanceof Number || defaultValue
instanceof Boolean)) {
--- End diff --
This will cause
[TestAvroReaderWithEmbeddedSchema](https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/test/java/org/apache/nifi/avro/TestAvroReaderWithEmbeddedSchema.java)
to fail.
---
[jira] [Commented] (NIFI-4893) Cannot convert Avro schemas to Record schemas with default value in arrays
[
https://issues.apache.org/jira/browse/NIFI-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370254#comment-16370254
]
ASF GitHub Bot commented on NIFI-4893:
--
Github user zenfenan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2480#discussion_r169385890
--- Diff:
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/RecordField.java
---
@@ -76,8 +77,12 @@ public RecordField(final String fieldName, final
DataType dataType, final Object
} else {
this.aliases = Collections.unmodifiableSet(aliases);
}
-
-this.defaultValue = defaultValue;
+if (RecordFieldType.ARRAY.equals(dataType.getFieldType()) &&
(defaultValue instanceof String
+|| defaultValue instanceof Number || defaultValue
instanceof Boolean)) {
--- End diff --
This will cause
[TestAvroReaderWithEmbeddedSchema](https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/test/java/org/apache/nifi/avro/TestAvroReaderWithEmbeddedSchema.java)
to fail.
> Cannot convert Avro schemas to Record schemas with default value in arrays
> --
>
> Key: NIFI-4893
> URL: https://issues.apache.org/jira/browse/NIFI-4893
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: ALL
>Reporter: Gardella Juan Pablo
>Priority: Major
> Attachments: issue1.zip
>
>
> Given an Avro Schema that has a default array defined, it is not possible to
> be converted to a Nifi Record Schema.
> To reproduce the bug, try to convert the following Avro schema to Record
> Schema:
> {code}
> {
> "type": "record",
> "name": "Foo1",
> "namespace": "foo.namespace",
> "fields": [
> {
> "name": "listOfInt",
> "type": {
> "type": "array",
> "items": "int"
> },
> "doc": "array of ints",
> "default": 0
> }
> ]
> }
> {code}
>
> Using org.apache.nifi.avro.AvroTypeUtil class. Attached a maven project to
> reproduce the issue and also the fix.
> * To reproduce the bug, run "mvn clean test"
> * To test the fix, run "mvn clean test -Ppatch".
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4894) nf-canvas-utils#queryBulletins does not account for possible proxy path
[ https://issues.apache.org/jira/browse/NIFI-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370220#comment-16370220 ] ASF GitHub Bot commented on NIFI-4894: -- Github user scottyaslan commented on the issue: https://github.com/apache/nifi/pull/2482 Reviewing... > nf-canvas-utils#queryBulletins does not account for possible proxy path > --- > > Key: NIFI-4894 > URL: https://issues.apache.org/jira/browse/NIFI-4894 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.5.0 >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Critical > > The queryBulletins function in nf-canvas-utils does not account for possible > proxy paths when querying for bulletins. We cannot, unfortunately, use > relative paths here since we need to know the full URL (to know it's length) > in order to break the query into multiple requests if necessary. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2482: NIFI-4894: Ensuring that any proxy paths are retained when...
Github user scottyaslan commented on the issue: https://github.com/apache/nifi/pull/2482 Reviewing... ---
[GitHub] nifi pull request #2180: Added GetMongoAggregation to support running Mongo ...
Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2180#discussion_r169360501 --- Diff: nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/resources/docs/org.apache.nifi.processors.mongodb.RunMongoAggregation/additionalDetails.html --- @@ -0,0 +1,44 @@ + + + + + +GetKafka --- End diff -- Should be RunMongoAggregation, not that you'd ever really see the title of this page. I'll update on merge ---
[GitHub] nifi pull request #2180: Added GetMongoAggregation to support running Mongo ...
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2180#discussion_r169371909
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/RunMongoAggregation.java
---
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.nifi.processors.mongodb;
+
+import com.mongodb.BasicDBObject;
+import com.mongodb.client.AggregateIterable;
+import com.mongodb.client.MongoCollection;
+import com.mongodb.client.MongoCursor;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.bson.conversions.Bson;
+import com.fasterxml.jackson.databind.ObjectMapper;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+@Tags({"mongo", "aggregation", "aggregate"})
+@CapabilityDescription("A processor that runs an aggregation query
whenever a flowfile is received.")
+@InputRequirement(InputRequirement.Requirement.INPUT_ALLOWED)
+@EventDriven
+public class RunMongoAggregation extends AbstractMongoProcessor {
+
+private final static Set relationships;
+private final static List propertyDescriptors;
+
+static final Relationship REL_ORIGINAL = new Relationship.Builder()
+.description("The input flowfile gets sent to this
relationship when the query succeeds.")
+.name("original")
+.build();
+static final Relationship REL_FAILURE = new Relationship.Builder()
+.description("The input flowfile gets sent to this
relationship when the query fails.")
+.name("failure")
+.build();
+static final Relationship REL_RESULTS = new Relationship.Builder()
+.description("The result set of the aggregation will be sent
to this relationship.")
+.name("results")
+.build();
+
+static final List buildAggregationQuery(String query) throws
IOException {
+List result = new ArrayList<>();
+
+ObjectMapper mapper = new ObjectMapper();
+List values = mapper.readValue(query, List.class);
+for (Map val : values) {
+result.add(new BasicDBObject(val));
+}
+
+return result;
+}
+
+public static final Validator AGG_VALIDATOR = (subject, value,
context) -> {
+final ValidationResult.Builder builder = new
ValidationResult.Builder();
+builder.subject(subject).input(value);
+
+if (context.isExpressionLanguageSupported(subject) &&
context.isExpressionLanguagePresent(value)) {
+return builder.valid(true).explanation("Contains Expression
Language").build();
+}
+
+String reason = null;
+try {
+buildAggregationQuery(value);
+} catch (final RuntimeException | IOException e) {
+reason = e.getLocalizedMessage();
+}
+
+return builder.explanation(reason).valid(reason == null).build();
+};
+
+static final PropertyDescriptor QUERY = new
PropertyDescriptor.Builder()
+.name("mongo-a
[jira] [Updated] (MINIFICPP-408) Secure minifi controller
[ https://issues.apache.org/jira/browse/MINIFICPP-408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] marco polo updated MINIFICPP-408: - Status: Patch Available (was: Open) > Secure minifi controller > > > Key: MINIFICPP-408 > URL: https://issues.apache.org/jira/browse/MINIFICPP-408 > Project: NiFi MiNiFi C++ > Issue Type: Sub-task >Reporter: marco polo >Assignee: marco polo >Priority: Major > > MiNiFi Controller only works with non secure sockets. Give users the option > to secure their instance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4894) nf-canvas-utils#queryBulletins does not account for possible proxy path
[ https://issues.apache.org/jira/browse/NIFI-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-4894: -- Affects Version/s: 1.5.0 Status: Patch Available (was: In Progress) > nf-canvas-utils#queryBulletins does not account for possible proxy path > --- > > Key: NIFI-4894 > URL: https://issues.apache.org/jira/browse/NIFI-4894 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.5.0 >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Critical > > The queryBulletins function in nf-canvas-utils does not account for possible > proxy paths when querying for bulletins. We cannot, unfortunately, use > relative paths here since we need to know the full URL (to know it's length) > in order to break the query into multiple requests if necessary. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4894) nf-canvas-utils#queryBulletins does not account for possible proxy path
[ https://issues.apache.org/jira/browse/NIFI-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370141#comment-16370141 ] ASF GitHub Bot commented on NIFI-4894: -- GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/2482 NIFI-4894: Ensuring that any proxy paths are retained when querying for bulletins NIFI-4894: - Ensuring that any proxy paths are retained when querying for bulletins. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-4894 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2482.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2482 commit c463f78b12f569a6954ce66e79b709e4805986f5 Author: Matt Gilman Date: 2018-02-20T15:28:50Z NIFI-4894: - Ensuring that any proxy paths are retained when querying for bulletins. > nf-canvas-utils#queryBulletins does not account for possible proxy path > --- > > Key: NIFI-4894 > URL: https://issues.apache.org/jira/browse/NIFI-4894 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Critical > > The queryBulletins function in nf-canvas-utils does not account for possible > proxy paths when querying for bulletins. We cannot, unfortunately, use > relative paths here since we need to know the full URL (to know it's length) > in order to break the query into multiple requests if necessary. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2482: NIFI-4894: Ensuring that any proxy paths are retain...
GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/2482 NIFI-4894: Ensuring that any proxy paths are retained when querying for bulletins NIFI-4894: - Ensuring that any proxy paths are retained when querying for bulletins. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-4894 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2482.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2482 commit c463f78b12f569a6954ce66e79b709e4805986f5 Author: Matt Gilman Date: 2018-02-20T15:28:50Z NIFI-4894: - Ensuring that any proxy paths are retained when querying for bulletins. ---
[jira] [Assigned] (NIFI-4894) nf-canvas-utils#queryBulletins does not account for possible proxy path
[ https://issues.apache.org/jira/browse/NIFI-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman reassigned NIFI-4894: - Assignee: Matt Gilman > nf-canvas-utils#queryBulletins does not account for possible proxy path > --- > > Key: NIFI-4894 > URL: https://issues.apache.org/jira/browse/NIFI-4894 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Critical > > The queryBulletins function in nf-canvas-utils does not account for possible > proxy paths when querying for bulletins. We cannot, unfortunately, use > relative paths here since we need to know the full URL (to know it's length) > in order to break the query into multiple requests if necessary. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-409) Remove unused artifacts from Configuration Listener removal
[ https://issues.apache.org/jira/browse/MINIFICPP-409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370134#comment-16370134 ] ASF GitHub Bot commented on MINIFICPP-409: -- Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/267 +1 pending travis successdoh > Remove unused artifacts from Configuration Listener removal > --- > > Key: MINIFICPP-409 > URL: https://issues.apache.org/jira/browse/MINIFICPP-409 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.4.0 >Reporter: Aldrin Piri >Assignee: Aldrin Piri >Priority: Minor > Fix For: 0.5.0 > > > With the introduction of C2 and related efforts, some artifacts were left > behind when the Configuration Listener was removed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp issue #267: MINIFICPP-409 Removing unused constants relating...
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/267 +1 pending travis successdoh ---
[jira] [Commented] (MINIFICPP-409) Remove unused artifacts from Configuration Listener removal
[ https://issues.apache.org/jira/browse/MINIFICPP-409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370131#comment-16370131 ] ASF GitHub Bot commented on MINIFICPP-409: -- GitHub user apiri opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/267 MINIFICPP-409 Removing unused constants relating to Configuration Listener MINIFICPP-409 Removing unused constants relating to Configuration Listener and its security. Removing these items from the README. Replacing stubbed out values in minifi.properties to reflect currently used keys. You can merge this pull request into a Git repository by running: $ git pull https://github.com/apiri/nifi-minifi-cpp MINIFICPP-409 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/267.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #267 commit 814b77ff5f60f7b69490a2980b1bc0588f63ff8e Author: Aldrin Piri Date: 2018-02-20T15:14:18Z MINIFICPP-409 Removing unused constants relating to Configuration Listener and its security. Removing these items from the README. Replacing stubbed out values in minifi.properties to reflect currently used keys. > Remove unused artifacts from Configuration Listener removal > --- > > Key: MINIFICPP-409 > URL: https://issues.apache.org/jira/browse/MINIFICPP-409 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.4.0 >Reporter: Aldrin Piri >Assignee: Aldrin Piri >Priority: Minor > Fix For: 0.5.0 > > > With the introduction of C2 and related efforts, some artifacts were left > behind when the Configuration Listener was removed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #267: MINIFICPP-409 Removing unused constants r...
GitHub user apiri opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/267 MINIFICPP-409 Removing unused constants relating to Configuration Listener MINIFICPP-409 Removing unused constants relating to Configuration Listener and its security. Removing these items from the README. Replacing stubbed out values in minifi.properties to reflect currently used keys. You can merge this pull request into a Git repository by running: $ git pull https://github.com/apiri/nifi-minifi-cpp MINIFICPP-409 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/267.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #267 commit 814b77ff5f60f7b69490a2980b1bc0588f63ff8e Author: Aldrin Piri Date: 2018-02-20T15:14:18Z MINIFICPP-409 Removing unused constants relating to Configuration Listener and its security. Removing these items from the README. Replacing stubbed out values in minifi.properties to reflect currently used keys. ---
[jira] [Updated] (MINIFICPP-409) Remove unused artifacts from Configuration Listener removal
[ https://issues.apache.org/jira/browse/MINIFICPP-409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aldrin Piri updated MINIFICPP-409: -- Fix Version/s: 0.5.0 > Remove unused artifacts from Configuration Listener removal > --- > > Key: MINIFICPP-409 > URL: https://issues.apache.org/jira/browse/MINIFICPP-409 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.4.0 >Reporter: Aldrin Piri >Assignee: Aldrin Piri >Priority: Minor > Fix For: 0.5.0 > > > With the introduction of C2 and related efforts, some artifacts were left > behind when the Configuration Listener was removed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (MINIFICPP-409) Remove unused artifacts from Configuration Listener removal
[ https://issues.apache.org/jira/browse/MINIFICPP-409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aldrin Piri updated MINIFICPP-409: -- Status: Patch Available (was: Open) > Remove unused artifacts from Configuration Listener removal > --- > > Key: MINIFICPP-409 > URL: https://issues.apache.org/jira/browse/MINIFICPP-409 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.4.0 >Reporter: Aldrin Piri >Assignee: Aldrin Piri >Priority: Minor > > With the introduction of C2 and related efforts, some artifacts were left > behind when the Configuration Listener was removed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MINIFICPP-409) Remove unused artifacts from Configuration Listener removal
Aldrin Piri created MINIFICPP-409: - Summary: Remove unused artifacts from Configuration Listener removal Key: MINIFICPP-409 URL: https://issues.apache.org/jira/browse/MINIFICPP-409 Project: NiFi MiNiFi C++ Issue Type: Improvement Affects Versions: 0.4.0 Reporter: Aldrin Piri Assignee: Aldrin Piri With the introduction of C2 and related efforts, some artifacts were left behind when the Configuration Listener was removed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-4897) Add documentation to user guide for terminating a processor
Mark Payne created NIFI-4897: Summary: Add documentation to user guide for terminating a processor Key: NIFI-4897 URL: https://issues.apache.org/jira/browse/NIFI-4897 Project: Apache NiFi Issue Type: Sub-task Components: Documentation & Website Reporter: Mark Payne -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-4896) Add option to UI for terminating a Processor when stopped but still has threads
Mark Payne created NIFI-4896: Summary: Add option to UI for terminating a Processor when stopped but still has threads Key: NIFI-4896 URL: https://issues.apache.org/jira/browse/NIFI-4896 Project: Apache NiFi Issue Type: Sub-task Components: Core UI Reporter: Mark Payne -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-4895) Implement back-end for terminating a Processor
Mark Payne created NIFI-4895: Summary: Implement back-end for terminating a Processor Key: NIFI-4895 URL: https://issues.apache.org/jira/browse/NIFI-4895 Project: Apache NiFi Issue Type: Sub-task Components: Core Framework Reporter: Mark Payne Assignee: Mark Payne -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-4894) nf-canvas-utils#queryBulletins does not account for possible proxy path
Matt Gilman created NIFI-4894: - Summary: nf-canvas-utils#queryBulletins does not account for possible proxy path Key: NIFI-4894 URL: https://issues.apache.org/jira/browse/NIFI-4894 Project: Apache NiFi Issue Type: Bug Components: Core UI Reporter: Matt Gilman The queryBulletins function in nf-canvas-utils does not account for possible proxy paths when querying for bulletins. We cannot, unfortunately, use relative paths here since we need to know the full URL (to know it's length) in order to break the query into multiple requests if necessary. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4367) InvokedScriptedProcessor does not support scripted processor that extends AbstractProcessor
[ https://issues.apache.org/jira/browse/NIFI-4367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370090#comment-16370090 ] Marco Gaido edited comment on NIFI-4367 at 2/20/18 2:14 PM: [~frett27] sorry did you have time to check the PR I sent you? Any luck going on with this? Thanks. was (Author: mgaido): [~frett27] sorry did you have time to check the PR I sent you? Any luck going on with this? > InvokedScriptedProcessor does not support scripted processor that extends > AbstractProcessor > --- > > Key: NIFI-4367 > URL: https://issues.apache.org/jira/browse/NIFI-4367 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Patrice Freydiere >Priority: Major > Labels: InvokeScriptedProcessor, validation > > InvokeScriptedProcessor pass his ValidationContext to the inner script, > validate call > InvokeScriptedProcessor Line 465 :final > Collection instanceResults = instance.validate(context); > > The problem is that the invokedscript pass the ScriptFile PropertyDescriptor > that is validated, if the script derived from the > AbstractConfigurableComponent, (because the AbstractConfigurableComponent > validate all the context properties). > The context should be refined to remove the InvokeScriptedProcessor > Properties. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4367) InvokedScriptedProcessor does not support scripted processor that extends AbstractProcessor
[ https://issues.apache.org/jira/browse/NIFI-4367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370090#comment-16370090 ] Marco Gaido commented on NIFI-4367: --- [~frett27] sorry did you have time to check the PR I sent you? Any luck going on with this? > InvokedScriptedProcessor does not support scripted processor that extends > AbstractProcessor > --- > > Key: NIFI-4367 > URL: https://issues.apache.org/jira/browse/NIFI-4367 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Patrice Freydiere >Priority: Major > Labels: InvokeScriptedProcessor, validation > > InvokeScriptedProcessor pass his ValidationContext to the inner script, > validate call > InvokeScriptedProcessor Line 465 :final > Collection instanceResults = instance.validate(context); > > The problem is that the invokedscript pass the ScriptFile PropertyDescriptor > that is validated, if the script derived from the > AbstractConfigurableComponent, (because the AbstractConfigurableComponent > validate all the context properties). > The context should be refined to remove the InvokeScriptedProcessor > Properties. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-406) Change StreamFactory to use SSLContextService if one is available.
[ https://issues.apache.org/jira/browse/MINIFICPP-406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370070#comment-16370070 ] ASF GitHub Bot commented on MINIFICPP-406: -- Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/265 Adding https://issues.apache.org/jira/browse/MINIFICPP-408 to this > Change StreamFactory to use SSLContextService if one is available. > --- > > Key: MINIFICPP-406 > URL: https://issues.apache.org/jira/browse/MINIFICPP-406 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: marco polo >Assignee: marco polo >Priority: Major > > Currently socket coordination is done through minifi.properties and is > unaware of any context service. We should change this to make configuration > consistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp issue #265: MINIFICPP-406: Ensure that Context Service OR mi...
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/265 Adding https://issues.apache.org/jira/browse/MINIFICPP-408 to this ---
[jira] [Created] (MINIFICPP-408) Secure minifi controller
marco polo created MINIFICPP-408: Summary: Secure minifi controller Key: MINIFICPP-408 URL: https://issues.apache.org/jira/browse/MINIFICPP-408 Project: NiFi MiNiFi C++ Issue Type: Sub-task Reporter: marco polo Assignee: marco polo MiNiFi Controller only works with non secure sockets. Give users the option to secure their instance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4891) Nifi flow version control should ignore (unused) remote ports in RPGs
[ https://issues.apache.org/jira/browse/NIFI-4891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370060#comment-16370060 ] Mark Payne commented on NIFI-4891: -- [~zeophlite] thanks for reporting this! We have addressed this issue, in the same way that that you describe, on the master branch. This work was done in NIFI-4863. Many thanks for the very detailed description of how to replicate! That always makes it far easier not only to identify the issue but to address it as well! > Nifi flow version control should ignore (unused) remote ports in RPGs > - > > Key: NIFI-4891 > URL: https://issues.apache.org/jira/browse/NIFI-4891 > Project: Apache NiFi > Issue Type: Bug > Components: Flow Versioning >Affects Versions: 1.5.0 >Reporter: Daniel Skates >Priority: Major > > Nifi flow version control is inconsistent in tracking actions for Remote > Process Groups > > Suggested solution: ignore changes to unused remote ports (i.e. ports that > have not been switched on in the "Mange remote ports" dialog. This also > prevents unrelated input/output ports from affecting a version controlled > flow. > > See below for list of steps to reproduce, note that it creates an > inconsistent state between PG1 and PG2 > # start Nifi > # create "group-a" Process Group (call it PG1) > # enter PG1 > # add RPG to http://localhost:8080/nifi/ HTTP > # leave PG1 > # commit changes to flow control > # add input port "input-a" > # add funnel > # add connection "input-a" to funnel > # start "input-a" > # enter group-a > # refresh remote on RPG > # new port appears when calling "Mange remote ports", no change to version > control for PG1 > # drag in new PG, import "group-a" (call it PG2) > # immediately PG2 detects local change: "Remote Input Port was added" > # repeat 7-10 to add new Input Port "input-b" > # no change for PG1 > # PG2 detects local change: "Remote Input Port was added" > The situation is worse if you attempt to commit changes to PG2 > > # commit changes to PG2 > # refresh canvas, PG1 will detect new version > # update PG1 to new version > # PG1 will immediately detects local changes: "Remote Input Port was > added/removed" > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (NIFI-4891) Nifi flow version control should ignore (unused) remote ports in RPGs
[ https://issues.apache.org/jira/browse/NIFI-4891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne resolved NIFI-4891. -- Resolution: Fixed > Nifi flow version control should ignore (unused) remote ports in RPGs > - > > Key: NIFI-4891 > URL: https://issues.apache.org/jira/browse/NIFI-4891 > Project: Apache NiFi > Issue Type: Bug > Components: Flow Versioning >Affects Versions: 1.5.0 >Reporter: Daniel Skates >Priority: Major > > Nifi flow version control is inconsistent in tracking actions for Remote > Process Groups > > Suggested solution: ignore changes to unused remote ports (i.e. ports that > have not been switched on in the "Mange remote ports" dialog. This also > prevents unrelated input/output ports from affecting a version controlled > flow. > > See below for list of steps to reproduce, note that it creates an > inconsistent state between PG1 and PG2 > # start Nifi > # create "group-a" Process Group (call it PG1) > # enter PG1 > # add RPG to http://localhost:8080/nifi/ HTTP > # leave PG1 > # commit changes to flow control > # add input port "input-a" > # add funnel > # add connection "input-a" to funnel > # start "input-a" > # enter group-a > # refresh remote on RPG > # new port appears when calling "Mange remote ports", no change to version > control for PG1 > # drag in new PG, import "group-a" (call it PG2) > # immediately PG2 detects local change: "Remote Input Port was added" > # repeat 7-10 to add new Input Port "input-b" > # no change for PG1 > # PG2 detects local change: "Remote Input Port was added" > The situation is worse if you attempt to commit changes to PG2 > > # commit changes to PG2 > # refresh canvas, PG1 will detect new version > # update PG1 to new version > # PG1 will immediately detects local changes: "Remote Input Port was > added/removed" > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-407) Change wording
[ https://issues.apache.org/jira/browse/MINIFICPP-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370001#comment-16370001 ] ASF GitHub Bot commented on MINIFICPP-407: -- GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/266 MINIFICPP-407: Update readme to clarify that we use semver and will n… …ot break api and abi compatibility Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-407 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/266.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #266 commit 574538e9e88cf1405b8b6e6c784e614ebaddc968 Author: Marc Parisi Date: 2018-02-20T12:31:38Z MINIFICPP-407: Update readme to clarify that we use semver and will not break api and abi compatibility > Change wording > -- > > Key: MINIFICPP-407 > URL: https://issues.apache.org/jira/browse/MINIFICPP-407 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: marco polo >Assignee: marco polo >Priority: Major > Fix For: 0.5.0 > > > Change wording of Readme to tell that we are GA -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #266: MINIFICPP-407: Update readme to clarify t...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/266 MINIFICPP-407: Update readme to clarify that we use semver and will n⦠â¦ot break api and abi compatibility Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-407 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/266.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #266 commit 574538e9e88cf1405b8b6e6c784e614ebaddc968 Author: Marc Parisi Date: 2018-02-20T12:31:38Z MINIFICPP-407: Update readme to clarify that we use semver and will not break api and abi compatibility ---
[jira] [Created] (MINIFICPP-407) Change wording
marco polo created MINIFICPP-407: Summary: Change wording Key: MINIFICPP-407 URL: https://issues.apache.org/jira/browse/MINIFICPP-407 Project: NiFi MiNiFi C++ Issue Type: Improvement Reporter: marco polo Assignee: marco polo Fix For: 0.5.0 Change wording of Readme to tell that we are GA -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4893) Cannot convert Avro schemas to Record schemas with default value in arrays
[
https://issues.apache.org/jira/browse/NIFI-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gardella Juan Pablo updated NIFI-4893:
--
Summary: Cannot convert Avro schemas to Record schemas with default value
in arrays (was: Cannot convert Avro schemas to Record schemas with default
arrays)
> Cannot convert Avro schemas to Record schemas with default value in arrays
> --
>
> Key: NIFI-4893
> URL: https://issues.apache.org/jira/browse/NIFI-4893
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: ALL
>Reporter: Gardella Juan Pablo
>Priority: Major
> Attachments: issue1.zip
>
>
> Given an Avro Schema that has a default array defined, it is not possible to
> be converted to a Nifi Record Schema.
> To reproduce the bug, try to convert the following Avro schema to Record
> Schema:
> {code}
> {
> "type": "record",
> "name": "Foo1",
> "namespace": "foo.namespace",
> "fields": [
> {
> "name": "listOfInt",
> "type": {
> "type": "array",
> "items": "int"
> },
> "doc": "array of ints",
> "default": 0
> }
> ]
> }
> {code}
>
> Using org.apache.nifi.avro.AvroTypeUtil class. Attached a maven project to
> reproduce the issue and also the fix.
> * To reproduce the bug, run "mvn clean test"
> * To test the fix, run "mvn clean test -Ppatch".
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4892) ValidateCSV Processor - no doublequote escaping in "invalid" output
[ https://issues.apache.org/jira/browse/NIFI-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369966#comment-16369966 ] Eugeny Kolpakov commented on NIFI-4892: --- Hi [~pvillard]! Thanks for quick response - I've added a couple comments/suggestions on the PR - hope I'm not abusing the hospitality :) > ValidateCSV Processor - no doublequote escaping in "invalid" output > --- > > Key: NIFI-4892 > URL: https://issues.apache.org/jira/browse/NIFI-4892 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.4.0, 1.5.0 >Reporter: Eugeny Kolpakov >Assignee: Pierre Villard >Priority: Major > Attachments: ValidateCSV_LineByLine_EscapingBug.xml > > > ValidateCSV processor fails to preserve field escaping for flow files routed > to "invalid" relation when "Line by Line" validation strategy is used. > > Minimal sample flow attached. > > Observed in 1.4.0, but 1.5.0 might be affected as well - I'll update this > ticket after upgrading to 1.5.0 (unless someone could do that earlier) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4892) ValidateCSV Processor - no doublequote escaping in "invalid" output
[
https://issues.apache.org/jira/browse/NIFI-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369964#comment-16369964
]
ASF GitHub Bot commented on NIFI-4892:
--
Github user e-kolpakov commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2481#discussion_r169291719
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestValidateCsv.java
---
@@ -333,4 +333,23 @@ public void testMultipleRuns() {
runner.assertTransferCount(ValidateCsv.REL_VALID, 2);
runner.assertTransferCount(ValidateCsv.REL_INVALID, 0);
}
+
+@Test
+public void testEscapingLineByLine() {
+final TestRunner runner = TestRunners.newTestRunner(new
ValidateCsv());
+runner.setProperty(ValidateCsv.DELIMITER_CHARACTER, ",");
+runner.setProperty(ValidateCsv.END_OF_LINE_CHARACTER, "\r\n");
+runner.setProperty(ValidateCsv.QUOTE_CHARACTER, "\"");
+runner.setProperty(ValidateCsv.HEADER, "false");
+runner.setProperty(ValidateCsv.VALIDATION_STRATEGY,
ValidateCsv.VALIDATE_LINES_INDIVIDUALLY);
+
+runner.setProperty(ValidateCsv.SCHEMA,
"ParseInt(),ParseInt(),ParseInt()");
+
+runner.enqueue("Field1,\"Field2,excaped\",Field3");
--- End diff --
@pvillard31 Also, I've observed roughly the same behavior with "valid"
output, but couldn't distill it to a minimal example. Maybe worth
adding/extending a test for "valid" as well?
> ValidateCSV Processor - no doublequote escaping in "invalid" output
> ---
>
> Key: NIFI-4892
> URL: https://issues.apache.org/jira/browse/NIFI-4892
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.4.0, 1.5.0
>Reporter: Eugeny Kolpakov
>Assignee: Pierre Villard
>Priority: Major
> Attachments: ValidateCSV_LineByLine_EscapingBug.xml
>
>
> ValidateCSV processor fails to preserve field escaping for flow files routed
> to "invalid" relation when "Line by Line" validation strategy is used.
>
> Minimal sample flow attached.
>
> Observed in 1.4.0, but 1.5.0 might be affected as well - I'll update this
> ticket after upgrading to 1.5.0 (unless someone could do that earlier)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4892) ValidateCSV Processor - no doublequote escaping in "invalid" output
[
https://issues.apache.org/jira/browse/NIFI-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369963#comment-16369963
]
ASF GitHub Bot commented on NIFI-4892:
--
Github user e-kolpakov commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2481#discussion_r169291484
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestValidateCsv.java
---
@@ -333,4 +333,23 @@ public void testMultipleRuns() {
runner.assertTransferCount(ValidateCsv.REL_VALID, 2);
runner.assertTransferCount(ValidateCsv.REL_INVALID, 0);
}
+
+@Test
+public void testEscapingLineByLine() {
+final TestRunner runner = TestRunners.newTestRunner(new
ValidateCsv());
+runner.setProperty(ValidateCsv.DELIMITER_CHARACTER, ",");
+runner.setProperty(ValidateCsv.END_OF_LINE_CHARACTER, "\r\n");
+runner.setProperty(ValidateCsv.QUOTE_CHARACTER, "\"");
+runner.setProperty(ValidateCsv.HEADER, "false");
+runner.setProperty(ValidateCsv.VALIDATION_STRATEGY,
ValidateCsv.VALIDATE_LINES_INDIVIDUALLY);
+
+runner.setProperty(ValidateCsv.SCHEMA,
"ParseInt(),ParseInt(),ParseInt()");
+
+runner.enqueue("Field1,\"Field2,excaped\",Field3");
--- End diff --
@pvillard31 Suggestion: extract into `final String` and use in an
assertion? I.e.
```
final String payload = "Field1,\"Field2,escaped\",Field3"
runner.enqueue(payload)
...
runner.getFlowFilesForRelationship(ValidateCsv.REL_INVALID).get(0).assertContentEquals(payload)
```
Also, `excaped` is definitely a typo in my template, no need to repeat it
here.
> ValidateCSV Processor - no doublequote escaping in "invalid" output
> ---
>
> Key: NIFI-4892
> URL: https://issues.apache.org/jira/browse/NIFI-4892
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.4.0, 1.5.0
>Reporter: Eugeny Kolpakov
>Assignee: Pierre Villard
>Priority: Major
> Attachments: ValidateCSV_LineByLine_EscapingBug.xml
>
>
> ValidateCSV processor fails to preserve field escaping for flow files routed
> to "invalid" relation when "Line by Line" validation strategy is used.
>
> Minimal sample flow attached.
>
> Observed in 1.4.0, but 1.5.0 might be affected as well - I'll update this
> ticket after upgrading to 1.5.0 (unless someone could do that earlier)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2481: NIFI-4892 - ValidateCSV: no doublequote escaping in...
Github user e-kolpakov commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2481#discussion_r169291719
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestValidateCsv.java
---
@@ -333,4 +333,23 @@ public void testMultipleRuns() {
runner.assertTransferCount(ValidateCsv.REL_VALID, 2);
runner.assertTransferCount(ValidateCsv.REL_INVALID, 0);
}
+
+@Test
+public void testEscapingLineByLine() {
+final TestRunner runner = TestRunners.newTestRunner(new
ValidateCsv());
+runner.setProperty(ValidateCsv.DELIMITER_CHARACTER, ",");
+runner.setProperty(ValidateCsv.END_OF_LINE_CHARACTER, "\r\n");
+runner.setProperty(ValidateCsv.QUOTE_CHARACTER, "\"");
+runner.setProperty(ValidateCsv.HEADER, "false");
+runner.setProperty(ValidateCsv.VALIDATION_STRATEGY,
ValidateCsv.VALIDATE_LINES_INDIVIDUALLY);
+
+runner.setProperty(ValidateCsv.SCHEMA,
"ParseInt(),ParseInt(),ParseInt()");
+
+runner.enqueue("Field1,\"Field2,excaped\",Field3");
--- End diff --
@pvillard31 Also, I've observed roughly the same behavior with "valid"
output, but couldn't distill it to a minimal example. Maybe worth
adding/extending a test for "valid" as well?
---
[GitHub] nifi pull request #2481: NIFI-4892 - ValidateCSV: no doublequote escaping in...
Github user e-kolpakov commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2481#discussion_r169291484
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestValidateCsv.java
---
@@ -333,4 +333,23 @@ public void testMultipleRuns() {
runner.assertTransferCount(ValidateCsv.REL_VALID, 2);
runner.assertTransferCount(ValidateCsv.REL_INVALID, 0);
}
+
+@Test
+public void testEscapingLineByLine() {
+final TestRunner runner = TestRunners.newTestRunner(new
ValidateCsv());
+runner.setProperty(ValidateCsv.DELIMITER_CHARACTER, ",");
+runner.setProperty(ValidateCsv.END_OF_LINE_CHARACTER, "\r\n");
+runner.setProperty(ValidateCsv.QUOTE_CHARACTER, "\"");
+runner.setProperty(ValidateCsv.HEADER, "false");
+runner.setProperty(ValidateCsv.VALIDATION_STRATEGY,
ValidateCsv.VALIDATE_LINES_INDIVIDUALLY);
+
+runner.setProperty(ValidateCsv.SCHEMA,
"ParseInt(),ParseInt(),ParseInt()");
+
+runner.enqueue("Field1,\"Field2,excaped\",Field3");
--- End diff --
@pvillard31 Suggestion: extract into `final String` and use in an
assertion? I.e.
```
final String payload = "Field1,\"Field2,escaped\",Field3"
runner.enqueue(payload)
...
runner.getFlowFilesForRelationship(ValidateCsv.REL_INVALID).get(0).assertContentEquals(payload)
```
Also, `excaped` is definitely a typo in my template, no need to repeat it
here.
---
[jira] [Updated] (NIFI-4892) ValidateCSV Processor - no doublequote escaping in "invalid" output
[ https://issues.apache.org/jira/browse/NIFI-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-4892: - Status: Patch Available (was: Open) > ValidateCSV Processor - no doublequote escaping in "invalid" output > --- > > Key: NIFI-4892 > URL: https://issues.apache.org/jira/browse/NIFI-4892 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.5.0, 1.4.0 >Reporter: Eugeny Kolpakov >Assignee: Pierre Villard >Priority: Major > Attachments: ValidateCSV_LineByLine_EscapingBug.xml > > > ValidateCSV processor fails to preserve field escaping for flow files routed > to "invalid" relation when "Line by Line" validation strategy is used. > > Minimal sample flow attached. > > Observed in 1.4.0, but 1.5.0 might be affected as well - I'll update this > ticket after upgrading to 1.5.0 (unless someone could do that earlier) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4892) ValidateCSV Processor - no doublequote escaping in "invalid" output
[ https://issues.apache.org/jira/browse/NIFI-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369850#comment-16369850 ] Pierre Villard commented on NIFI-4892: -- Hi [~Kolpakov], Thanks for reporting this. I just had a look and added a unit test to confirm the issue. It seems to be on purpose while reading the columns: [https://github.com/super-csv/super-csv/blob/master/super-csv/src/main/java/org/supercsv/io/Tokenizer.java#L211] But I just submitted a PR that slightly changes the way we are dealing with individual invalid lines. This way it solves the issue you reported. > ValidateCSV Processor - no doublequote escaping in "invalid" output > --- > > Key: NIFI-4892 > URL: https://issues.apache.org/jira/browse/NIFI-4892 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.4.0, 1.5.0 >Reporter: Eugeny Kolpakov >Assignee: Pierre Villard >Priority: Major > Attachments: ValidateCSV_LineByLine_EscapingBug.xml > > > ValidateCSV processor fails to preserve field escaping for flow files routed > to "invalid" relation when "Line by Line" validation strategy is used. > > Minimal sample flow attached. > > Observed in 1.4.0, but 1.5.0 might be affected as well - I'll update this > ticket after upgrading to 1.5.0 (unless someone could do that earlier) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4892) ValidateCSV Processor - no doublequote escaping in "invalid" output
[ https://issues.apache.org/jira/browse/NIFI-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369849#comment-16369849 ] ASF GitHub Bot commented on NIFI-4892: -- GitHub user pvillard31 opened a pull request: https://github.com/apache/nifi/pull/2481 NIFI-4892 - ValidateCSV: no doublequote escaping in invalid output Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pvillard31/nifi NIFI-4892 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2481.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2481 commit 3a57674f30dc27eaf83e770814f6fe51d3be8b05 Author: Pierre Villard Date: 2018-02-20T09:36:31Z NIFI-4892 - ValidateCSV: no doublequote escaping in invalid output > ValidateCSV Processor - no doublequote escaping in "invalid" output > --- > > Key: NIFI-4892 > URL: https://issues.apache.org/jira/browse/NIFI-4892 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.4.0, 1.5.0 >Reporter: Eugeny Kolpakov >Assignee: Pierre Villard >Priority: Major > Attachments: ValidateCSV_LineByLine_EscapingBug.xml > > > ValidateCSV processor fails to preserve field escaping for flow files routed > to "invalid" relation when "Line by Line" validation strategy is used. > > Minimal sample flow attached. > > Observed in 1.4.0, but 1.5.0 might be affected as well - I'll update this > ticket after upgrading to 1.5.0 (unless someone could do that earlier) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2481: NIFI-4892 - ValidateCSV: no doublequote escaping in...
GitHub user pvillard31 opened a pull request: https://github.com/apache/nifi/pull/2481 NIFI-4892 - ValidateCSV: no doublequote escaping in invalid output Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pvillard31/nifi NIFI-4892 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2481.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2481 commit 3a57674f30dc27eaf83e770814f6fe51d3be8b05 Author: Pierre Villard Date: 2018-02-20T09:36:31Z NIFI-4892 - ValidateCSV: no doublequote escaping in invalid output ---
[jira] [Updated] (NIFI-4892) ValidateCSV Processor - no doublequote escaping in "invalid" output
[ https://issues.apache.org/jira/browse/NIFI-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-4892: - Affects Version/s: 1.5.0 > ValidateCSV Processor - no doublequote escaping in "invalid" output > --- > > Key: NIFI-4892 > URL: https://issues.apache.org/jira/browse/NIFI-4892 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.4.0, 1.5.0 >Reporter: Eugeny Kolpakov >Assignee: Pierre Villard >Priority: Major > Attachments: ValidateCSV_LineByLine_EscapingBug.xml > > > ValidateCSV processor fails to preserve field escaping for flow files routed > to "invalid" relation when "Line by Line" validation strategy is used. > > Minimal sample flow attached. > > Observed in 1.4.0, but 1.5.0 might be affected as well - I'll update this > ticket after upgrading to 1.5.0 (unless someone could do that earlier) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4892) ValidateCSV Processor - no doublequote escaping in "invalid" output
[ https://issues.apache.org/jira/browse/NIFI-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-4892: - Issue Type: Bug (was: Improvement) > ValidateCSV Processor - no doublequote escaping in "invalid" output > --- > > Key: NIFI-4892 > URL: https://issues.apache.org/jira/browse/NIFI-4892 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.4.0, 1.5.0 >Reporter: Eugeny Kolpakov >Assignee: Pierre Villard >Priority: Major > Attachments: ValidateCSV_LineByLine_EscapingBug.xml > > > ValidateCSV processor fails to preserve field escaping for flow files routed > to "invalid" relation when "Line by Line" validation strategy is used. > > Minimal sample flow attached. > > Observed in 1.4.0, but 1.5.0 might be affected as well - I'll update this > ticket after upgrading to 1.5.0 (unless someone could do that earlier) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (NIFI-4892) ValidateCSV Processor - no doublequote escaping in "invalid" output
[ https://issues.apache.org/jira/browse/NIFI-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard reassigned NIFI-4892: Assignee: Pierre Villard > ValidateCSV Processor - no doublequote escaping in "invalid" output > --- > > Key: NIFI-4892 > URL: https://issues.apache.org/jira/browse/NIFI-4892 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.4.0, 1.5.0 >Reporter: Eugeny Kolpakov >Assignee: Pierre Villard >Priority: Major > Attachments: ValidateCSV_LineByLine_EscapingBug.xml > > > ValidateCSV processor fails to preserve field escaping for flow files routed > to "invalid" relation when "Line by Line" validation strategy is used. > > Minimal sample flow attached. > > Observed in 1.4.0, but 1.5.0 might be affected as well - I'll update this > ticket after upgrading to 1.5.0 (unless someone could do that earlier) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4892) ValidateCSV Processor - no doublequote escaping in "invalid" output
[ https://issues.apache.org/jira/browse/NIFI-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-4892: - Component/s: Extensions > ValidateCSV Processor - no doublequote escaping in "invalid" output > --- > > Key: NIFI-4892 > URL: https://issues.apache.org/jira/browse/NIFI-4892 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.4.0, 1.5.0 >Reporter: Eugeny Kolpakov >Assignee: Pierre Villard >Priority: Major > Attachments: ValidateCSV_LineByLine_EscapingBug.xml > > > ValidateCSV processor fails to preserve field escaping for flow files routed > to "invalid" relation when "Line by Line" validation strategy is used. > > Minimal sample flow attached. > > Observed in 1.4.0, but 1.5.0 might be affected as well - I'll update this > ticket after upgrading to 1.5.0 (unless someone could do that earlier) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
