[GitHub] nifi pull request #2091: NIFI-3484 GenerateTableFetch Should Allow for Right...
GitHub user pvillard31 opened a pull request: https://github.com/apache/nifi/pull/2091 NIFI-3484 GenerateTableFetch Should Allow for Right Boundary To replace PR #1513 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pvillard31/nifi PR1513 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2091.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2091 commit dde372cf3e4179a4442088adbe9775108f7b Author: patricker Date: 2017-02-15T19:11:56Z NIFI-3484 commit e611456ba71a1ec7fd946fd3290cce2f1610aec0 Author: Pierre Villard Date: 2017-08-02T13:52:06Z fix checkstyle issue, and added unit test showing data duplication issue --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3484) GenerateTableFetch Should Allow for Right Boundary
[ https://issues.apache.org/jira/browse/NIFI-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128757#comment-16128757 ] ASF GitHub Bot commented on NIFI-3484: -- GitHub user pvillard31 opened a pull request: https://github.com/apache/nifi/pull/2091 NIFI-3484 GenerateTableFetch Should Allow for Right Boundary To replace PR #1513 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pvillard31/nifi PR1513 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2091.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2091 commit dde372cf3e4179a4442088adbe9775108f7b Author: patricker Date: 2017-02-15T19:11:56Z NIFI-3484 commit e611456ba71a1ec7fd946fd3290cce2f1610aec0 Author: Pierre Villard Date: 2017-08-02T13:52:06Z fix checkstyle issue, and added unit test showing data duplication issue > GenerateTableFetch Should Allow for Right Boundary > -- > > Key: NIFI-3484 > URL: https://issues.apache.org/jira/browse/NIFI-3484 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > > When using GenerateTableFetch it places no right hand boundary on pages of > data. This can lead to issues when the statement says to get the next 1000 > records greater then a specific key, but records were added to the table > between the time the processor executed and when the SQL is being executed. > As a result it pulls in records that did not exist when the processor was > run. On the next execution of the processor these records will be pulled in > a second time. > Example: > Partition Size = 1000 > First run (no state): Count(*)=4700 and MAX(ID)=4700. > 5 FlowFiles are generated, the last one will say to fetch 1000, not 700. (But > I don't think this is really a bug, just an observation). > 5 Flow Files are now in queue to be executed by ExecuteSQL. Before the 5th > file can execute 400 new rows are added to the table. When the final SQL > statement is executed 300 extra records, with higher ID values, will also be > pulled into NiFi. > Second run (state: ID=4700). Count(*) ID>4700 = 400 and MAX(ID)=5100. > 1 Flow File is generated, but includes 300 records already pulled into NiFI. > The solution is to have an optional property that will let users use the new > MAX(ID) as a right boundary when generating queries. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #1513: NIFI-3484 GenerateTableFetch Should Allow for Right Bounda...
Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/1513 No problem @patricker, just opened #2091. Let me know if it looks good to you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3484) GenerateTableFetch Should Allow for Right Boundary
[ https://issues.apache.org/jira/browse/NIFI-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128759#comment-16128759 ] ASF GitHub Bot commented on NIFI-3484: -- Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/1513 No problem @patricker, just opened #2091. Let me know if it looks good to you. > GenerateTableFetch Should Allow for Right Boundary > -- > > Key: NIFI-3484 > URL: https://issues.apache.org/jira/browse/NIFI-3484 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > > When using GenerateTableFetch it places no right hand boundary on pages of > data. This can lead to issues when the statement says to get the next 1000 > records greater then a specific key, but records were added to the table > between the time the processor executed and when the SQL is being executed. > As a result it pulls in records that did not exist when the processor was > run. On the next execution of the processor these records will be pulled in > a second time. > Example: > Partition Size = 1000 > First run (no state): Count(*)=4700 and MAX(ID)=4700. > 5 FlowFiles are generated, the last one will say to fetch 1000, not 700. (But > I don't think this is really a bug, just an observation). > 5 Flow Files are now in queue to be executed by ExecuteSQL. Before the 5th > file can execute 400 new rows are added to the table. When the final SQL > statement is executed 300 extra records, with higher ID values, will also be > pulled into NiFi. > Second run (state: ID=4700). Count(*) ID>4700 = 400 and MAX(ID)=5100. > 1 Flow File is generated, but includes 300 records already pulled into NiFI. > The solution is to have an optional property that will let users use the new > MAX(ID) as a right boundary when generating queries. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4295) When selecting a controller service for a processor, services that belong to the wrong scope are shown
[
https://issues.apache.org/jira/browse/NIFI-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128777#comment-16128777
]
ASF GitHub Bot commented on NIFI-4295:
--
Github user mcgilman commented on the issue:
https://github.com/apache/nifi/pull/2087
Will review...
> When selecting a controller service for a processor, services that belong to
> the wrong scope are shown
> --
>
> Key: NIFI-4295
> URL: https://issues.apache.org/jira/browse/NIFI-4295
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.4.0
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.4.0
>
>
> I have a Process Group, and in that Process Group I have defined a Controller
> Service. If I then move up to the root group and add a Processor, that takes
> a Controller Service of that type, I can now choose the Controller Service
> that is defined in a child group. When I do so, my processor becomes invalid
> and the reason given for the processor to be invalid is a NullPointer:
> {code}
> 2017-08-14 12:38:24,016 WARN [NiFi Web Server-76]
> o.a.n.controller.StandardProcessorNode Failed during validation
> java.lang.NullPointerException: null
> at
> org.apache.nifi.processor.StandardValidationContext.isValidationRequired(StandardValidationContext.java:143)
> at
> org.apache.nifi.components.PropertyDescriptor.validate(PropertyDescriptor.java:150)
> at
> org.apache.nifi.components.AbstractConfigurableComponent.validate(AbstractConfigurableComponent.java:103)
> at
> org.apache.nifi.controller.AbstractConfiguredComponent.validate(AbstractConfiguredComponent.java:329)
> at
> org.apache.nifi.controller.StandardProcessorNode.isValid(StandardProcessorNode.java:968)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatus(FlowController.java:2964)
> at
> org.apache.nifi.controller.FlowController.getGroupStatus(FlowController.java:2559)
> at
> org.apache.nifi.controller.FlowController.getGroupStatus(FlowController.java:2518)
> at
> org.apache.nifi.controller.FlowController.getGroupStatus(FlowController.java:2485)
> at
> org.apache.nifi.web.controller.ControllerFacade.getProcessGroupStatus(ControllerFacade.java:599)
> at
> org.apache.nifi.web.controller.ControllerFacade$$FastClassBySpringCGLIB$$5a42ba54.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
> at
> org.apache.nifi.web.controller.ControllerFacade$$EnhancerBySpringCGLIB$$344e3ba2.getProcessGroupStatus()
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.getProcessGroupFlow(StandardNiFiServiceFacade.java:3054)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
> at
> org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithReadLock(NiFiServiceFacadeLock.java:137)
> at
> org.apache.nifi.web.NiFiServiceFacadeLock.getLock(NiFiServiceFacadeLock.java:108)
> at sun.reflect.GeneratedMethodAccessor168.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
> at
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68)
> at
> org.springframework.aop.framework.ReflectiveMethodI
[GitHub] nifi issue #2087: NIFI-4295: When determining which controller services to r...
Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2087 Will review... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #2055: NIFI-4028 - fix cache update when Wait releases flow files
Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/2055 Thanks @ijokarumawak for your review and work, that's much cleaner with your commit. I just added it to this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4028) Wait processor removes cache regardless of the remaing counter
[
https://issues.apache.org/jira/browse/NIFI-4028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128783#comment-16128783
]
ASF GitHub Bot commented on NIFI-4028:
--
Github user pvillard31 commented on the issue:
https://github.com/apache/nifi/pull/2055
Thanks @ijokarumawak for your review and work, that's much cleaner with
your commit. I just added it to this PR.
> Wait processor removes cache regardless of the remaing counter
> --
>
> Key: NIFI-4028
> URL: https://issues.apache.org/jira/browse/NIFI-4028
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Koji Kawamura
>Assignee: Pierre Villard
> Attachments: WaitNotify_NIFI-4028.xml
>
>
> Current [Wait processor implementation removes cache
> entry|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/Wait.java#L438]
> if it's configured with 'Releasable FlowFile Count' is one, when it finds a
> wait signal reaches to 'Target Signal Count' no matter how large the current
> count is.
> This behavior can be problematic, when an user expects the same Signal
> Identifier to be released multiple FlowFiles one by one while multiple counts
> to be accumulated by Notify.
> For example this scenario does not work:
> {code}
> 1. Notify notifies to a 'signal-a' + 1 => 'signal-a' = 1
> 2. Notify notifies to a 'signal-a' again, +1 => 'signal-a' = 2
> 3. Wait releases 1 FlowFile, then remove the 'signal-a' => 'signal-a' is
> removed
> 4. Wait will not be able to release another FlowFile
> {code}
> This should be fixed as follows:
> {code}
> 1. Notify notifies to a 'signal-a' + 1 => 'signal-a' = 1
> 2. Notify notifies to a 'signal-a' again, +1 => 'signal-a' = 2
> 3. Wait releases 1 FlowFile, then update the 'signal-a', -1 => 'signal-a' = 1
> 4. Wait releases another FlowFile, then 'signal-a' becomes 0, remove
> 'signal-a'
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] nifi pull request #2080: NIFI-4062 Provide an option to disable DTD validati...
Github user arunma commented on a diff in the pull request: https://github.com/apache/nifi/pull/2080#discussion_r133464470 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestEvaluateXPath.java --- @@ -16,23 +16,23 @@ */ package org.apache.nifi.processors.standard; -import static org.junit.Assert.assertTrue; +import org.apache.nifi.util.MockFlowFile; +import org.apache.nifi.util.TestRunner; +import org.apache.nifi.util.TestRunners; +import org.junit.Test; +import javax.xml.xpath.XPathFactoryConfigurationException; import java.io.IOException; import java.nio.file.Path; import java.nio.file.Paths; -import javax.xml.xpath.XPathFactoryConfigurationException; - -import org.apache.nifi.util.MockFlowFile; -import org.apache.nifi.util.TestRunner; -import org.apache.nifi.util.TestRunners; - -import org.junit.Test; +import static org.junit.Assert.assertTrue; --- End diff -- :-) Absolutely. Fixed it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #2080: NIFI-4062 Provide an option to disable DTD validati...
Github user arunma commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2080#discussion_r133464617
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestEvaluateXQuery.java
---
@@ -65,7 +65,6 @@ public void testSetTransformerProperties() throws
Exception {
}
}
-@Ignore("this test is failing")
--- End diff --
Anytime !
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (NIFI-4062) Provide an option to disable DTD validation for EvaluateXPath and EvaluateXQuery
[
https://issues.apache.org/jira/browse/NIFI-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128863#comment-16128863
]
ASF GitHub Bot commented on NIFI-4062:
--
Github user arunma commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2080#discussion_r133464617
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestEvaluateXQuery.java
---
@@ -65,7 +65,6 @@ public void testSetTransformerProperties() throws
Exception {
}
}
-@Ignore("this test is failing")
--- End diff --
Anytime !
> Provide an option to disable DTD validation for EvaluateXPath and
> EvaluateXQuery
>
>
> Key: NIFI-4062
> URL: https://issues.apache.org/jira/browse/NIFI-4062
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.0.0
>Reporter: Koji Kawamura
>Assignee: Arun Manivannan
> Labels: beginner
>
> Currently EvaluateXPath and EvaluateXQuery can fail if the source document
> has DOCTYPE definition in it, especially using external DTD. It would be
> useful if these processor provides a property to disable DTD validation when
> evaluate XPath or XQuery.
> References:
> https://community.hortonworks.com/questions/107121/does-evaluatexpath-support-doctype.html
> https://stackoverflow.com/questions/243728/how-to-disable-dtd-at-runtime-in-javas-xpath
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4062) Provide an option to disable DTD validation for EvaluateXPath and EvaluateXQuery
[ https://issues.apache.org/jira/browse/NIFI-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128864#comment-16128864 ] ASF GitHub Bot commented on NIFI-4062: -- Github user arunma commented on a diff in the pull request: https://github.com/apache/nifi/pull/2080#discussion_r133464579 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestEvaluateXQuery.java --- @@ -31,19 +35,15 @@ import java.util.Map.Entry; import java.util.Properties; -import javax.xml.transform.OutputKeys; -import javax.xml.xpath.XPathFactoryConfigurationException; - -import org.apache.nifi.util.MockFlowFile; -import org.apache.nifi.util.TestRunner; -import org.apache.nifi.util.TestRunners; -import org.junit.Ignore; - -import org.junit.Test; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; --- End diff -- Fixed it. > Provide an option to disable DTD validation for EvaluateXPath and > EvaluateXQuery > > > Key: NIFI-4062 > URL: https://issues.apache.org/jira/browse/NIFI-4062 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Arun Manivannan > Labels: beginner > > Currently EvaluateXPath and EvaluateXQuery can fail if the source document > has DOCTYPE definition in it, especially using external DTD. It would be > useful if these processor provides a property to disable DTD validation when > evaluate XPath or XQuery. > References: > https://community.hortonworks.com/questions/107121/does-evaluatexpath-support-doctype.html > https://stackoverflow.com/questions/243728/how-to-disable-dtd-at-runtime-in-javas-xpath -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4062) Provide an option to disable DTD validation for EvaluateXPath and EvaluateXQuery
[
https://issues.apache.org/jira/browse/NIFI-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128865#comment-16128865
]
ASF GitHub Bot commented on NIFI-4062:
--
Github user arunma commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2080#discussion_r133464300
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EvaluateXPath.java
---
@@ -219,6 +234,25 @@ public void onTrigger(final ProcessContext context,
final ProcessSession session
}
final ComponentLog logger = getLogger();
+final XMLReader xmlReader;
+
+try {
+xmlReader = XMLReaderFactory.createXMLReader();
+} catch (SAXException e) {
+logger.error("Error while constructing XMLReader {}", new
Object[]{e});
+session.transfer(flowFiles, REL_FAILURE);
--- End diff --
Yikes. My bad ! Fixed it. Both throw ProcessException.
> Provide an option to disable DTD validation for EvaluateXPath and
> EvaluateXQuery
>
>
> Key: NIFI-4062
> URL: https://issues.apache.org/jira/browse/NIFI-4062
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.0.0
>Reporter: Koji Kawamura
>Assignee: Arun Manivannan
> Labels: beginner
>
> Currently EvaluateXPath and EvaluateXQuery can fail if the source document
> has DOCTYPE definition in it, especially using external DTD. It would be
> useful if these processor provides a property to disable DTD validation when
> evaluate XPath or XQuery.
> References:
> https://community.hortonworks.com/questions/107121/does-evaluatexpath-support-doctype.html
> https://stackoverflow.com/questions/243728/how-to-disable-dtd-at-runtime-in-javas-xpath
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] nifi pull request #2080: NIFI-4062 Provide an option to disable DTD validati...
Github user arunma commented on a diff in the pull request: https://github.com/apache/nifi/pull/2080#discussion_r133464579 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestEvaluateXQuery.java --- @@ -31,19 +35,15 @@ import java.util.Map.Entry; import java.util.Properties; -import javax.xml.transform.OutputKeys; -import javax.xml.xpath.XPathFactoryConfigurationException; - -import org.apache.nifi.util.MockFlowFile; -import org.apache.nifi.util.TestRunner; -import org.apache.nifi.util.TestRunners; -import org.junit.Ignore; - -import org.junit.Test; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; --- End diff -- Fixed it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #2080: NIFI-4062 Provide an option to disable DTD validati...
Github user arunma commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2080#discussion_r133464300
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EvaluateXPath.java
---
@@ -219,6 +234,25 @@ public void onTrigger(final ProcessContext context,
final ProcessSession session
}
final ComponentLog logger = getLogger();
+final XMLReader xmlReader;
+
+try {
+xmlReader = XMLReaderFactory.createXMLReader();
+} catch (SAXException e) {
+logger.error("Error while constructing XMLReader {}", new
Object[]{e});
+session.transfer(flowFiles, REL_FAILURE);
--- End diff --
Yikes. My bad ! Fixed it. Both throw ProcessException.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (NIFI-4062) Provide an option to disable DTD validation for EvaluateXPath and EvaluateXQuery
[ https://issues.apache.org/jira/browse/NIFI-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128866#comment-16128866 ] ASF GitHub Bot commented on NIFI-4062: -- Github user arunma commented on a diff in the pull request: https://github.com/apache/nifi/pull/2080#discussion_r133464470 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestEvaluateXPath.java --- @@ -16,23 +16,23 @@ */ package org.apache.nifi.processors.standard; -import static org.junit.Assert.assertTrue; +import org.apache.nifi.util.MockFlowFile; +import org.apache.nifi.util.TestRunner; +import org.apache.nifi.util.TestRunners; +import org.junit.Test; +import javax.xml.xpath.XPathFactoryConfigurationException; import java.io.IOException; import java.nio.file.Path; import java.nio.file.Paths; -import javax.xml.xpath.XPathFactoryConfigurationException; - -import org.apache.nifi.util.MockFlowFile; -import org.apache.nifi.util.TestRunner; -import org.apache.nifi.util.TestRunners; - -import org.junit.Test; +import static org.junit.Assert.assertTrue; --- End diff -- :-) Absolutely. Fixed it. > Provide an option to disable DTD validation for EvaluateXPath and > EvaluateXQuery > > > Key: NIFI-4062 > URL: https://issues.apache.org/jira/browse/NIFI-4062 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Arun Manivannan > Labels: beginner > > Currently EvaluateXPath and EvaluateXQuery can fail if the source document > has DOCTYPE definition in it, especially using external DTD. It would be > useful if these processor provides a property to disable DTD validation when > evaluate XPath or XQuery. > References: > https://community.hortonworks.com/questions/107121/does-evaluatexpath-support-doctype.html > https://stackoverflow.com/questions/243728/how-to-disable-dtd-at-runtime-in-javas-xpath -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2056: NIFI-4262 - MergeContent - option to add merged uui...
Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2056#discussion_r133466975 --- Diff: nifi-nar-bundles/nifi-extension-utils/nifi-processor-utils/src/main/java/org/apache/nifi/processor/util/bin/BinFiles.java --- @@ -94,6 +96,16 @@ .addValidator(StandardValidators.createTimePeriodValidator(1, TimeUnit.SECONDS, Integer.MAX_VALUE, TimeUnit.SECONDS)) .build(); +public static final PropertyDescriptor ADD_MERGED_UUID = new PropertyDescriptor.Builder() --- End diff -- Agree, this is just adding configuration complexity for very little (none) benefit. I just pushed a commit to remove this parameter. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4262) MergeContent - option to add merged uuid in original flow files
[ https://issues.apache.org/jira/browse/NIFI-4262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128869#comment-16128869 ] ASF GitHub Bot commented on NIFI-4262: -- Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2056#discussion_r133466975 --- Diff: nifi-nar-bundles/nifi-extension-utils/nifi-processor-utils/src/main/java/org/apache/nifi/processor/util/bin/BinFiles.java --- @@ -94,6 +96,16 @@ .addValidator(StandardValidators.createTimePeriodValidator(1, TimeUnit.SECONDS, Integer.MAX_VALUE, TimeUnit.SECONDS)) .build(); +public static final PropertyDescriptor ADD_MERGED_UUID = new PropertyDescriptor.Builder() --- End diff -- Agree, this is just adding configuration complexity for very little (none) benefit. I just pushed a commit to remove this parameter. > MergeContent - option to add merged uuid in original flow files > --- > > Key: NIFI-4262 > URL: https://issues.apache.org/jira/browse/NIFI-4262 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard > > With the apparition of Wait/Notify processors it is now possible tackle new > challenges when it comes to synchronize the execution of different parts of > the workflow. > The objective here is the following: > Flow files are sent to a MergeContent processor. Merged flow files are then > sent to a processor A while original flow files are sent to a processor B. I > want to trigger processor B when and only when processor A has completed. > To use the Wait/Notify approach, a common attribute must be available to be > used as a signal in the distributed cache. This JIRA is about adding a > processor property allowing a user to add the UUID of the merged flow file as > a new attribute of all the original flow files that are constituting the > merged flow file. > The template attached to NIFI-4028 can be used for this use case. Note that > the fix for NIFI-4028 is needed to solve the use case described in this JIRA. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2056: NIFI-4262 - MergeContent - option to add merged uui...
Github user pvillard31 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2056#discussion_r133467718
--- Diff:
nifi-nar-bundles/nifi-extension-utils/nifi-processor-utils/src/main/java/org/apache/nifi/processor/util/bin/Bin.java
---
@@ -173,4 +174,19 @@ public long getBinAge() {
final long ageInNanos = System.nanoTime() - creationMomentEpochNs;
return TimeUnit.MILLISECONDS.convert(ageInNanos,
TimeUnit.NANOSECONDS);
}
+
+/**
+ * @return the UUID of the flow file generated by the merge of this
bin's content
+ */
+public String getUuid() {
+return uuid;
+}
+
+/**
+ * Set the UUID of the flow file generated by the merge of this bin's
content
+ * @param uuid UUID to set
+ */
+public void setUuid(String uuid) {
--- End diff --
I just had another look and I'm not sure to see a way to do it nicely. If
you have an idea, let me know, I'll be happy to give it a try.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (NIFI-4262) MergeContent - option to add merged uuid in original flow files
[
https://issues.apache.org/jira/browse/NIFI-4262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128874#comment-16128874
]
ASF GitHub Bot commented on NIFI-4262:
--
Github user pvillard31 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2056#discussion_r133467718
--- Diff:
nifi-nar-bundles/nifi-extension-utils/nifi-processor-utils/src/main/java/org/apache/nifi/processor/util/bin/Bin.java
---
@@ -173,4 +174,19 @@ public long getBinAge() {
final long ageInNanos = System.nanoTime() - creationMomentEpochNs;
return TimeUnit.MILLISECONDS.convert(ageInNanos,
TimeUnit.NANOSECONDS);
}
+
+/**
+ * @return the UUID of the flow file generated by the merge of this
bin's content
+ */
+public String getUuid() {
+return uuid;
+}
+
+/**
+ * Set the UUID of the flow file generated by the merge of this bin's
content
+ * @param uuid UUID to set
+ */
+public void setUuid(String uuid) {
--- End diff --
I just had another look and I'm not sure to see a way to do it nicely. If
you have an idea, let me know, I'll be happy to give it a try.
> MergeContent - option to add merged uuid in original flow files
> ---
>
> Key: NIFI-4262
> URL: https://issues.apache.org/jira/browse/NIFI-4262
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>
> With the apparition of Wait/Notify processors it is now possible tackle new
> challenges when it comes to synchronize the execution of different parts of
> the workflow.
> The objective here is the following:
> Flow files are sent to a MergeContent processor. Merged flow files are then
> sent to a processor A while original flow files are sent to a processor B. I
> want to trigger processor B when and only when processor A has completed.
> To use the Wait/Notify approach, a common attribute must be available to be
> used as a signal in the distributed cache. This JIRA is about adding a
> processor property allowing a user to add the UUID of the merged flow file as
> a new attribute of all the original flow files that are constituting the
> merged flow file.
> The template attached to NIFI-4028 can be used for this use case. Note that
> the fix for NIFI-4028 is needed to solve the use case described in this JIRA.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] nifi issue #1968: NIFI-3931 - Added EL to properties in SFTP transfer
Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/1968 Hey @ijokarumawak, I agree the IT class is not really helpful, it was just something I used to test the changes on my side. Happy to remove this file: just pushed a commit for that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3931) putSftp process port property should support for expression language
[ https://issues.apache.org/jira/browse/NIFI-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128880#comment-16128880 ] ASF GitHub Bot commented on NIFI-3931: -- Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/1968 Hey @ijokarumawak, I agree the IT class is not really helpful, it was just something I used to test the changes on my side. Happy to remove this file: just pushed a commit for that. > putSftp process port property should support for expression language > > > Key: NIFI-3931 > URL: https://issues.apache.org/jira/browse/NIFI-3931 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.2.0 >Reporter: Cheng Chin Tat >Assignee: Pierre Villard >Priority: Minor > Labels: easyfix > Attachments: TestFTP.xml > > > PutSftp Processor port property should support for expression language so > that dynamic port number can be pass to the processor during run time. > Rather than preset the port on design time. > This changes involve changing the PropertyDescriptor SFTP_PORT validator > StandardValidators.NON_NEGATIVE_INTEGER_VALIDATOR to > StandardValidators.NON_EMPTY_VALIDATOR and other codes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2080: NIFI-4062 Provide an option to disable DTD validati...
Github user arunma closed the pull request at: https://github.com/apache/nifi/pull/2080 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4062) Provide an option to disable DTD validation for EvaluateXPath and EvaluateXQuery
[ https://issues.apache.org/jira/browse/NIFI-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128919#comment-16128919 ] ASF GitHub Bot commented on NIFI-4062: -- Github user arunma closed the pull request at: https://github.com/apache/nifi/pull/2080 > Provide an option to disable DTD validation for EvaluateXPath and > EvaluateXQuery > > > Key: NIFI-4062 > URL: https://issues.apache.org/jira/browse/NIFI-4062 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Arun Manivannan > Labels: beginner > > Currently EvaluateXPath and EvaluateXQuery can fail if the source document > has DOCTYPE definition in it, especially using external DTD. It would be > useful if these processor provides a property to disable DTD validation when > evaluate XPath or XQuery. > References: > https://community.hortonworks.com/questions/107121/does-evaluatexpath-support-doctype.html > https://stackoverflow.com/questions/243728/how-to-disable-dtd-at-runtime-in-javas-xpath -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-1604) PutSolrContentStream does not evaluate expression attribute
[
https://issues.apache.org/jira/browse/NIFI-1604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128930#comment-16128930
]
ASF GitHub Bot commented on NIFI-1604:
--
GitHub user pvillard31 opened a pull request:
https://github.com/apache/nifi/pull/2092
NIFI-1604 - PutSolrContentStream - Evaluate stream path against flow file
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with NIFI- where is the JIRA number
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [ ] Is your initial contribution a single, squashed commit?
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file, including the main
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in
which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/pvillard31/nifi NIFI-1604
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/2092.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2092
commit d3bad2bd2218818db0971654cbbca83a4ec66c07
Author: Pierre Villard
Date: 2017-08-16T15:12:16Z
NIFI-1604 - PutSolrContentStream - Evaluate stream path against flow file
> PutSolrContentStream does not evaluate expression attribute
> ---
>
> Key: NIFI-1604
> URL: https://issues.apache.org/jira/browse/NIFI-1604
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 0.4.1
>Reporter: Adis Cesir
>Priority: Minor
>
> PutSolrContentStream does not evaluate user created attributes correctly in
> the Content Stream Path property.
> An example is: /update/extract?literal.id=${some_attribute}&commit=true
> some_attribute will not be evaluated correctly
> Workaround is to explicitly define the solr content stream properties as:
> literal.id =
> commit =
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] nifi pull request #2092: NIFI-1604 - PutSolrContentStream - Evaluate stream ...
GitHub user pvillard31 opened a pull request: https://github.com/apache/nifi/pull/2092 NIFI-1604 - PutSolrContentStream - Evaluate stream path against flow file Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pvillard31/nifi NIFI-1604 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2092.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2092 commit d3bad2bd2218818db0971654cbbca83a4ec66c07 Author: Pierre Villard Date: 2017-08-16T15:12:16Z NIFI-1604 - PutSolrContentStream - Evaluate stream path against flow file --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-1604) PutSolrContentStream does not evaluate expression attribute
[
https://issues.apache.org/jira/browse/NIFI-1604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-1604:
-
Assignee: Pierre Villard
Status: Patch Available (was: Open)
> PutSolrContentStream does not evaluate expression attribute
> ---
>
> Key: NIFI-1604
> URL: https://issues.apache.org/jira/browse/NIFI-1604
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 0.4.1
>Reporter: Adis Cesir
>Assignee: Pierre Villard
>Priority: Minor
>
> PutSolrContentStream does not evaluate user created attributes correctly in
> the Content Stream Path property.
> An example is: /update/extract?literal.id=${some_attribute}&commit=true
> some_attribute will not be evaluated correctly
> Workaround is to explicitly define the solr content stream properties as:
> literal.id =
> commit =
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] nifi pull request #2093: NIFI-4062 Provide an option to disable DTD validati...
GitHub user arunma opened a pull request: https://github.com/apache/nifi/pull/2093 NIFI-4062 Provide an option to disable DTD validation for EvaluateXPa… …th and EvaluateXQuery Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/arunma/nifi master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2093.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2093 commit d2032da711a3ee14ddc499baccefe486072b6c51 Author: Arun Manivannan Date: 2017-08-12T05:41:18Z NIFI-4062 Provide an option to disable DTD validation for EvaluateXPath and EvaluateXQuery --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-1604) PutSolrContentStream does not evaluate expression attribute
[
https://issues.apache.org/jira/browse/NIFI-1604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-1604:
-
Labels: easyfix (was: )
Priority: Trivial (was: Minor)
Component/s: (was: Core Framework)
Extensions
> PutSolrContentStream does not evaluate expression attribute
> ---
>
> Key: NIFI-1604
> URL: https://issues.apache.org/jira/browse/NIFI-1604
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 0.4.1
>Reporter: Adis Cesir
>Assignee: Pierre Villard
>Priority: Trivial
> Labels: easyfix
>
> PutSolrContentStream does not evaluate user created attributes correctly in
> the Content Stream Path property.
> An example is: /update/extract?literal.id=${some_attribute}&commit=true
> some_attribute will not be evaluated correctly
> Workaround is to explicitly define the solr content stream properties as:
> literal.id =
> commit =
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4062) Provide an option to disable DTD validation for EvaluateXPath and EvaluateXQuery
[ https://issues.apache.org/jira/browse/NIFI-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128933#comment-16128933 ] ASF GitHub Bot commented on NIFI-4062: -- GitHub user arunma opened a pull request: https://github.com/apache/nifi/pull/2093 NIFI-4062 Provide an option to disable DTD validation for EvaluateXPa… …th and EvaluateXQuery Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/arunma/nifi master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2093.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2093 commit d2032da711a3ee14ddc499baccefe486072b6c51 Author: Arun Manivannan Date: 2017-08-12T05:41:18Z NIFI-4062 Provide an option to disable DTD validation for EvaluateXPath and EvaluateXQuery > Provide an option to disable DTD validation for EvaluateXPath and > EvaluateXQuery > > > Key: NIFI-4062 > URL: https://issues.apache.org/jira/browse/NIFI-4062 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Arun Manivannan > Labels: beginner > > Currently EvaluateXPath and EvaluateXQuery can fail if the source document > has DOCTYPE definition in it, especially using external DTD. It would be > useful if these processor provides a property to disable DTD validation when > evaluate XPath or XQuery. > References: > https://community.hortonworks.com/questions/107121/does-evaluatexpath-support-doctype.html > https://stackoverflow.com/questions/243728/how-to-disable-dtd-at-runtime-in-javas-xpath -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2093: NIFI-4062 Provide an option to disable DTD validation for ...
Github user arunma commented on the issue: https://github.com/apache/nifi/pull/2093 @ijokarumawak Thanks for all the patience and very useful review. I learnt quite a lot. Is there a way to prevent the PR from closing itself automatically when I tried to reset my master with the latest of the upstream? I feel bad for opening up different PRs like this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4062) Provide an option to disable DTD validation for EvaluateXPath and EvaluateXQuery
[ https://issues.apache.org/jira/browse/NIFI-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128949#comment-16128949 ] ASF GitHub Bot commented on NIFI-4062: -- Github user arunma commented on the issue: https://github.com/apache/nifi/pull/2093 @ijokarumawak Thanks for all the patience and very useful review. I learnt quite a lot. Is there a way to prevent the PR from closing itself automatically when I tried to reset my master with the latest of the upstream? I feel bad for opening up different PRs like this. > Provide an option to disable DTD validation for EvaluateXPath and > EvaluateXQuery > > > Key: NIFI-4062 > URL: https://issues.apache.org/jira/browse/NIFI-4062 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Arun Manivannan > Labels: beginner > > Currently EvaluateXPath and EvaluateXQuery can fail if the source document > has DOCTYPE definition in it, especially using external DTD. It would be > useful if these processor provides a property to disable DTD validation when > evaluate XPath or XQuery. > References: > https://community.hortonworks.com/questions/107121/does-evaluatexpath-support-doctype.html > https://stackoverflow.com/questions/243728/how-to-disable-dtd-at-runtime-in-javas-xpath -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (NIFI-1639) Provenance Scrapping/ Saving
[ https://issues.apache.org/jira/browse/NIFI-1639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-1639. -- Resolution: Not A Problem Closing. This should probably be asked on the mailing list. To answer the question: using the Site-to-Site provenance reporting task is certainly the best approach here. A good starting point could be the template around provenance data conversion available on this page: https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates > Provenance Scrapping/ Saving > > > Key: NIFI-1639 > URL: https://issues.apache.org/jira/browse/NIFI-1639 > Project: Apache NiFi > Issue Type: Wish > Components: Examples > Environment: RHEL 7, Dell 730xd server >Reporter: John Teabo > > Does anyone have any examples of writing all provenance data to a SQL > database? Excluding the actual content of the file of course. I just want to > keep a record of all provenance events for a period of 5 years. It would be > really nice If I could just age off the content of the files after a period > of time and keep all provenance events for 5 years. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (NIFI-1643) FlowFileHandlingException should extend RuntimeException
[ https://issues.apache.org/jira/browse/NIFI-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-1643. -- Resolution: Fixed Fix Version/s: 1.0.0 > FlowFileHandlingException should extend RuntimeException > > > Key: NIFI-1643 > URL: https://issues.apache.org/jira/browse/NIFI-1643 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.5.1 >Reporter: Michael Moser >Priority: Minor > Fix For: 1.0.0 > > > Looking at nifi-api in org.apache.nifi.processor.exceptions, I see that > FlowFileAccessException and MissingFlowFileException extend RuntimeException. > But FlowFileHandlingException extends ProcessException. > Should FlowFileHandlingException extend RuntimeException like the others? > I ask because the latest best practice for try-catch handling of exceptions > thrown from ProcessSession inside onTrigger() is to catch ProcessException. > This allows the exceptions mentioned above to escape an onTrigger() to be > handled in the framework (which usually rolls back the session). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (NIFI-1659) Enhance EvaluateJsonPath to be able to evaluate an attribute value
[ https://issues.apache.org/jira/browse/NIFI-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-1659. -- Resolution: Duplicate Fix Version/s: 1.0.0 Closing. The feature is not directly available in the EvaluateJsonPath processor but the EL function provides the feature using processors such as UpdateAttribute, ReplaceText, etc. > Enhance EvaluateJsonPath to be able to evaluate an attribute value > -- > > Key: NIFI-1659 > URL: https://issues.apache.org/jira/browse/NIFI-1659 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 0.5.1 >Reporter: Christopher McDermott >Priority: Minor > Fix For: 1.0.0 > > > Today EvaluateJsonPath evaluates the content of a flow file. This request is > to enhance EvaluateJsonPath to evaluate either the flow file content, or an > attribute value. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2081: NIFI-3780 - Should be able to specify Service...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2081 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3780) Should be able to specify Service name when I create a new Controller Service within a Processor/Service configuration dialog
[ https://issues.apache.org/jira/browse/NIFI-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128989#comment-16128989 ] ASF subversion and git services commented on NIFI-3780: --- Commit 02c05bc2037ca2e4ee1850a8cb765d85a4f3b8a3 in nifi's branch refs/heads/master from [~yuri1969] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=02c05bc ] NIFI-3780 - Should be able to specify Service... ...name when I create a new Controller Service within a Processor/Service configuration dialog Changed CS name default's behavior according to PR feedback. Signed-off-by: Scott Aslan This closes #2081 > Should be able to specify Service name when I create a new Controller Service > within a Processor/Service configuration dialog > - > > Key: NIFI-3780 > URL: https://issues.apache.org/jira/browse/NIFI-3780 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Mark Payne > > When I am configuring a Processor (or a Controller Service) and a Property > Descriptor allows me to create a new Controller Service, that New Controller > Service dialog allows me to choose which service type to create. However, we > often have a few different instances of the same Controller Service type. In > such a case, it is then confusing which service I'm looking at. > I would like that New Controller Service dialog to have a field for > specifying the name of the Controller Service. That field could either be > pre-populated with the class name and allow me to change it, or the field > could be blank and the class name can simply be used if nothing is entered. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-3780) Should be able to specify Service name when I create a new Controller Service within a Processor/Service configuration dialog
[ https://issues.apache.org/jira/browse/NIFI-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128990#comment-16128990 ] ASF GitHub Bot commented on NIFI-3780: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2081 > Should be able to specify Service name when I create a new Controller Service > within a Processor/Service configuration dialog > - > > Key: NIFI-3780 > URL: https://issues.apache.org/jira/browse/NIFI-3780 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Mark Payne > > When I am configuring a Processor (or a Controller Service) and a Property > Descriptor allows me to create a new Controller Service, that New Controller > Service dialog allows me to choose which service type to create. However, we > often have a few different instances of the same Controller Service type. In > such a case, it is then confusing which service I'm looking at. > I would like that New Controller Service dialog to have a field for > specifying the name of the Controller Service. That field could either be > pre-populated with the class name and allow me to change it, or the field > could be blank and the class name can simply be used if nothing is entered. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-3780) Should be able to specify Service name when I create a new Controller Service within a Processor/Service configuration dialog
[ https://issues.apache.org/jira/browse/NIFI-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128988#comment-16128988 ] ASF subversion and git services commented on NIFI-3780: --- Commit 79300de7a9866993a2e401ac7580892b7ce1512c in nifi's branch refs/heads/master from [~yuri1969] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=79300de ] NIFI-3780 - Should be able to specify Service... ...name when I create a new Controller Service within a Processor/Service configuration dialog I'm not sure how to integrate/style the text box. Suggestions are welcome. > Should be able to specify Service name when I create a new Controller Service > within a Processor/Service configuration dialog > - > > Key: NIFI-3780 > URL: https://issues.apache.org/jira/browse/NIFI-3780 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Mark Payne > > When I am configuring a Processor (or a Controller Service) and a Property > Descriptor allows me to create a new Controller Service, that New Controller > Service dialog allows me to choose which service type to create. However, we > often have a few different instances of the same Controller Service type. In > such a case, it is then confusing which service I'm looking at. > I would like that New Controller Service dialog to have a field for > specifying the name of the Controller Service. That field could either be > pre-populated with the class name and allow me to change it, or the field > could be blank and the class name can simply be used if nothing is entered. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-minifi-cpp pull request #128: MINIFI-376 removed defunct references to ...
GitHub user achristianson opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/128 MINIFI-376 removed defunct references to curlbuild.h ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFI-376 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/128.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #128 commit a037064623bae520ab6d510653ec4b36256f82f5 Author: Andrew I. Christianson Date: 2017-08-16T15:56:34Z MINIFI-376 removed defunct references to curlbuild.h --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #2081: NIFI-3780 - Should be able to specify Service...
Github user scottyaslan commented on the issue: https://github.com/apache/nifi/pull/2081 Thanks @yuri1969 this has been merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (NIFI-3780) Should be able to specify Service name when I create a new Controller Service within a Processor/Service configuration dialog
[ https://issues.apache.org/jira/browse/NIFI-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan resolved NIFI-3780. --- Resolution: Fixed > Should be able to specify Service name when I create a new Controller Service > within a Processor/Service configuration dialog > - > > Key: NIFI-3780 > URL: https://issues.apache.org/jira/browse/NIFI-3780 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Mark Payne > > When I am configuring a Processor (or a Controller Service) and a Property > Descriptor allows me to create a new Controller Service, that New Controller > Service dialog allows me to choose which service type to create. However, we > often have a few different instances of the same Controller Service type. In > such a case, it is then confusing which service I'm looking at. > I would like that New Controller Service dialog to have a field for > specifying the name of the Controller Service. That field could either be > pre-populated with the class name and allow me to change it, or the field > could be blank and the class name can simply be used if nothing is entered. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-3780) Should be able to specify Service name when I create a new Controller Service within a Processor/Service configuration dialog
[ https://issues.apache.org/jira/browse/NIFI-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128992#comment-16128992 ] ASF GitHub Bot commented on NIFI-3780: -- Github user scottyaslan commented on the issue: https://github.com/apache/nifi/pull/2081 Thanks @yuri1969 this has been merged to master. > Should be able to specify Service name when I create a new Controller Service > within a Processor/Service configuration dialog > - > > Key: NIFI-3780 > URL: https://issues.apache.org/jira/browse/NIFI-3780 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Mark Payne > > When I am configuring a Processor (or a Controller Service) and a Property > Descriptor allows me to create a new Controller Service, that New Controller > Service dialog allows me to choose which service type to create. However, we > often have a few different instances of the same Controller Service type. In > such a case, it is then confusing which service I'm looking at. > I would like that New Controller Service dialog to have a field for > specifying the name of the Controller Service. That field could either be > pre-populated with the class name and allow me to change it, or the field > could be blank and the class name can simply be used if nothing is entered. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (NIFI-1716) Implement a SplitCsv processor, possibly also a GetCSV
[ https://issues.apache.org/jira/browse/NIFI-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-1716. -- Resolution: Duplicate Fix Version/s: 1.2.0 > Implement a SplitCsv processor, possibly also a GetCSV > -- > > Key: NIFI-1716 > URL: https://issues.apache.org/jira/browse/NIFI-1716 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Dmitry Goldenberg > Fix For: 1.2.0 > > > I'm proposing a SplitCSV processor dedicated specifically to splitting CSV > content which is assumed to be in the flowfile-content of its incoming > flowfiles. > It appears that the current mode of splitting a CSV file is by using the > SplitText processor. However, it'd be great to have a CSV splitter to read > CSV records one by one and use the header row's header names to convert each > record into a FlowFile, with attributes set to correspond to the headers. > Whether or not the first row is a header should be a boolean configuration > option. In the absence of a header row, some sensible default column names > should be utilized, for example, one convention could be: column1, column2, > column3, etc. (or a naming strategy could be provided by the user in the > configuration). > Another option on the splitter needs to be the delimiter character (defaulted > to comma). > Empty lines shall be skipped from processing. > Extracted cell values shall be (optionally) whitespace-trimmed. > Jagged rows must have some sensible handling: > 1) For a given row, if there are fewer cells than in the header row, cells > shall be assigned to columns left to right, and any missing cells shall be > considered empty. > 2) For a given row, if there are more cells than in the header row, a > (non-fatal) error shall be generated for the row and the row shall be dropped > from processing. > As typically done with CSV, delimiter characters are ignored within quotes. > Elements may span multiple lines by having embedded carriage returns; such > elements must be quoted. > NIFI-1280 asks for a way to specify which columns are to be kept or skipped. > I'm proposing that instead of a separate processor, this would be implemented > as a configuration option on SplitCSV (a list of 0-based indices of columns > that are to be kept). > It may also make sense to expose a GetCSV ingress component which would share > most of its functionality with SplitCSV. Perhaps it's easiest if users just > follow a GetFile with SplitCSV, however in some cases it makes sense to save > on reading the file into a flowfile-content but rather process all CSV data > in-place, within a GetCSV. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (NIFI-1853) Newly created HBase client service cannot be chosen within a PutHBaseJSON processor
[
https://issues.apache.org/jira/browse/NIFI-1853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-1853.
--
Resolution: Cannot Reproduce
Cannot reproduce in 1.x line.
Note that controller services must be created at process group level to be used
in processors.
> Newly created HBase client service cannot be chosen within a PutHBaseJSON
> processor
> ---
>
> Key: NIFI-1853
> URL: https://issues.apache.org/jira/browse/NIFI-1853
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 0.5.0
> Environment: Operating system: CentOS7 (a VM created on OpenStack)
> Nifi cluster: 2VMs
>Reporter: Massimiliano Nigrelli
>Priority: Minor
>
> I create an HBase Client Service via the Controller Settings UI. It perfectly
> works. I then create a PutHbaseJSON processor and try to select the already
> created Hbase Client Service in the HBase Client Service field.
> The drop-down menu only lists "new value" and "create new service" (and there
> is no chance to select the already created one).
> So I click on "create new service" but in the only menu of the pop-up I
> cannot select the already created Hbase Client Service; the only chance I
> have is to create a brand new one (that is only available for Node and not
> for Cluster Manager).
> Issue a) I create an HBase Client Service and I cannot select it in the
> PutHbaseJSON processor
> Issue b) If I create an HBase Client Service from the "create new service"
> link, I cannot assign the availability ("Node" is the unmodifiable value).
> Hope it helps.
> Regards,
> Massimiliano
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (NIFI-1873) Reverse order of bulletins in Tooltip
[ https://issues.apache.org/jira/browse/NIFI-1873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-1873. -- Resolution: Fixed > Reverse order of bulletins in Tooltip > - > > Key: NIFI-1873 > URL: https://issues.apache.org/jira/browse/NIFI-1873 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Brandon DeVries >Priority: Minor > > When hovering over a bulletin on a processor, the bulletins are shown with > the most recent at the bottom... which can be obscured if the processor is > near the bottom of the screen. It would be helpful to have the most recent > bulletin at the top, so that it is the most visible. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (NIFI-1936) Testing processors requiring multiple incoming flowfiles
[ https://issues.apache.org/jira/browse/NIFI-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-1936. -- Resolution: Cannot Reproduce > Testing processors requiring multiple incoming flowfiles > > > Key: NIFI-1936 > URL: https://issues.apache.org/jira/browse/NIFI-1936 > Project: Apache NiFi > Issue Type: Bug > Components: Tools and Build >Affects Versions: 1.0.0, 0.6.1 >Reporter: Daniel Cave >Priority: Minor > > JUnit processor testing is failing in cases where a processor requires two > incoming flowfiles. The test runner seems unable to properly enqueue > multiple flowfiles at the same time. One of the flowfiles will have > attributes on it, the other will have null values. See > TestConvertDynamicJsonToAvro on NIFI-1935. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-minifi-cpp issue #128: MINIFI-376 removed defunct references to curlbui...
Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/128 reviewing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-4198) *ElasticsearchHttp processors do not expose Proxy settings
[ https://issues.apache.org/jira/browse/NIFI-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Manivannan updated NIFI-4198: -- Attachment: NIFI-4198-ElasticProxy.xml NIFI-4198-ElasticProxy-TestFixtures.pdf > *ElasticsearchHttp processors do not expose Proxy settings > -- > > Key: NIFI-4198 > URL: https://issues.apache.org/jira/browse/NIFI-4198 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Andre F de Miranda >Assignee: Arun Manivannan > Attachments: NIFI-4198-ElasticProxy-TestFixtures.pdf, > NIFI-4198-ElasticProxy.xml > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (NIFI-1953) object not serializable (class: org.apache.nifi.spark.NiFiReceiver$ReceiveRunnable$1, value:
[ https://issues.apache.org/jira/browse/NIFI-1953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-1953. -- Resolution: Cannot Reproduce I've not been able to reproduce the issue while testing with more recent versions of the dependencies. > object not serializable (class: > org.apache.nifi.spark.NiFiReceiver$ReceiveRunnable$1, value: > - > > Key: NIFI-1953 > URL: https://issues.apache.org/jira/browse/NIFI-1953 > Project: Apache NiFi > Issue Type: Bug >Reporter: pradeep arumalla > > > when I use StorageLevel.MEMORY_AND_DISK() , throws the below exception in > Apache Spark.Not always I can use MEMORY_ONLY(), when there is too much data > coming in , I will be getting "Could not compute split, block > input-0-1464774108087 not found" exceptions if I use > StorageLevel.MEMORY_ONLY(). > JavaReceiverInputDStream packetStream = > ssc.receiverStream(new NiFiReceiver(config, > StorageLevel.MEMORY_AND_DISK())); > I get below exception > 16/06/01 12:50:28 ERROR scheduler.ReceiverTracker: Deregistered receiver for > stream 0: Restarting receiver with delay 2000ms: Failed to receive data from > NiFi - java.io.NotSerializableException: > org.apache.nifi.spark.NiFiReceiver$ReceiveRunnable$1 > Serialization stack: > - object not serializable (class: > org.apache.nifi.spark.NiFiReceiver$ReceiveRunnable$1, value: > org.apache.nifi.spark.NiFiReceiver$ReceiveRunnable$1@70fb979) > at > org.apache.spark.serializer.SerializationDebugger$.improveException(SerializationDebugger.scala:40) > > at > org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:47) > > at > org.apache.spark.serializer.SerializationStream.writeAll(Serializer.scala:153) > > at > org.apache.spark.storage.BlockManager.dataSerializeStream(BlockManager.scala:1189) > > at > org.apache.spark.storage.BlockManager.dataSerialize(BlockManager.scala:1198) > at > org.apache.spark.storage.MemoryStore.putArray(MemoryStore.scala:131) > at > org.apache.spark.storage.MemoryStore.putIterator(MemoryStore.scala:168) > at > org.apache.spark.storage.MemoryStore.putIterator(MemoryStore.scala:142) > at > org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:790) > at > org.apache.spark.storage.BlockManager.putIterator(BlockManager.scala:637) > at > org.apache.spark.streaming.receiver.BlockManagerBasedBlockHandler.storeBlock(ReceivedBlockHandler.scala:81) > > at > org.apache.spark.streaming.receiver.ReceiverSupervisorImpl.pushAndReportBlock(ReceiverSupervisorImpl.scala:141) > > at > org.apache.spark.streaming.receiver.ReceiverSupervisorImpl.pushIterator(ReceiverSupervisorImpl.scala:121) > > at > org.apache.spark.streaming.receiver.Receiver.store(Receiver.scala:152) > at > org.apache.nifi.spark.NiFiReceiver$ReceiveRunnable.run(NiFiReceiver.java:182) > at java.lang.Thread.run(Thread.java:745) > Remove Ads -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (NIFI-1970) Memorialize Apache NiFi SWAG logo
[ https://issues.apache.org/jira/browse/NIFI-1970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-1970. -- Resolution: Delivered > Memorialize Apache NiFi SWAG logo > - > > Key: NIFI-1970 > URL: https://issues.apache.org/jira/browse/NIFI-1970 > Project: Apache NiFi > Issue Type: Task >Reporter: Joseph Witt > Attachments: logo-nifi.ai, logo-nifi-drop.ai > > > Rob Moran made this image. It was used to order apache nifi tshirts and > stickers which I'm documenting on the wiki. Using this to hold the image > file. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (NIFI-1986) Expression Language not supported for Mongo Collection Names
[ https://issues.apache.org/jira/browse/NIFI-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-1986. -- Resolution: Duplicate > Expression Language not supported for Mongo Collection Names > > > Key: NIFI-1986 > URL: https://issues.apache.org/jira/browse/NIFI-1986 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 0.6.1 >Reporter: jon schendt >Priority: Minor > Labels: mongodb, nifi, putmongo > Original Estimate: 4h > Remaining Estimate: 4h > > When creating a workflow to insert data into Mongo, I have many data flows > running into the PutMongo processor. I would like to insert these records > into collections that show their data source via an attribute. The PutMongo > processor (which is driven by AbstractMongoProcessor) doesn't allow for > dynamic collection assignment via expression language. The workaround is > having 1..n PutMongo processors, all pointing to the same DB, with different > collection names assigned for each workflow. This seems very un-Nifi, and > appears to be fairly simple to modify the AbstractMongoProcessor to allow > expressions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2094: NIFI-4198 *ElasticsearchHttp processors do not expo...
GitHub user arunma opened a pull request: https://github.com/apache/nifi/pull/2094 NIFI-4198 *ElasticsearchHttp processors do not expose Proxy settings Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/arunma/nifi NIFI-4198_ES_Squashed Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2094.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2094 commit e4dd50eb1c9ac587c2a78d0dbccee6596e28067f Author: Arun Manivannan Date: 2017-08-16T17:30:05Z NIFI-4198 *ElasticsearchHttp processors do not expose Proxy settings --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #2020: [NiFi-3973] Add PutKudu Processor for ingesting data to Ku...
Github user rickysaltzer commented on the issue: https://github.com/apache/nifi/pull/2020 @cammachusa I'm having a hard time getting your new commits...as well as even understanding how I got them in the first place. @joewitt am I missing something regarding getting the new commits? I'm running a fetch on the repo located here on github. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4198) *ElasticsearchHttp processors do not expose Proxy settings
[ https://issues.apache.org/jira/browse/NIFI-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129126#comment-16129126 ] ASF GitHub Bot commented on NIFI-4198: -- GitHub user arunma opened a pull request: https://github.com/apache/nifi/pull/2094 NIFI-4198 *ElasticsearchHttp processors do not expose Proxy settings Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/arunma/nifi NIFI-4198_ES_Squashed Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2094.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2094 commit e4dd50eb1c9ac587c2a78d0dbccee6596e28067f Author: Arun Manivannan Date: 2017-08-16T17:30:05Z NIFI-4198 *ElasticsearchHttp processors do not expose Proxy settings > *ElasticsearchHttp processors do not expose Proxy settings > -- > > Key: NIFI-4198 > URL: https://issues.apache.org/jira/browse/NIFI-4198 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Andre F de Miranda >Assignee: Arun Manivannan > Attachments: NIFI-4198-ElasticProxy-TestFixtures.pdf, > NIFI-4198-ElasticProxy.xml > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4198) *ElasticsearchHttp processors do not expose Proxy settings
[ https://issues.apache.org/jira/browse/NIFI-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Manivannan updated NIFI-4198: -- Affects Version/s: 1.4.0 Status: Patch Available (was: In Progress) > *ElasticsearchHttp processors do not expose Proxy settings > -- > > Key: NIFI-4198 > URL: https://issues.apache.org/jira/browse/NIFI-4198 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Andre F de Miranda >Assignee: Arun Manivannan > Attachments: NIFI-4198-ElasticProxy-TestFixtures.pdf, > NIFI-4198-ElasticProxy.xml > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (NIFI-2004) NiFi should not allow creation of templates with the same name
[ https://issues.apache.org/jira/browse/NIFI-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-2004. -- Resolution: Duplicate Fix Version/s: 1.0.0 > NiFi should not allow creation of templates with the same name > --- > > Key: NIFI-2004 > URL: https://issues.apache.org/jira/browse/NIFI-2004 > Project: Apache NiFi > Issue Type: Bug >Reporter: Oleg Zhurakousky > Fix For: 1.0.0 > > > NiFi allows creation of multiple templates with the same name regardless > wether such templates are created from the same or different flows -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2020: [NiFi-3973] Add PutKudu Processor for ingesting data to Ku...
Github user rickysaltzer commented on the issue: https://github.com/apache/nifi/pull/2020 False alarm, I got it now :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #128: MINIFI-376 removed defunct references to ...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/128 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #2020: [NiFi-3973] Add PutKudu Processor for ingesting data to Ku...
Github user cammachusa commented on the issue: https://github.com/apache/nifi/pull/2020 @rickysaltzer , hahaha. When will you close this PR, I am waiting for your review every day :=) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #2087: NIFI-4295: When determining which controller services to r...
Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2087 @markap14 It appears the changes introduced are causing some test failures. ``` Failed tests: TestStandardControllerServiceProvider.validateEnableServices:506 null TestStandardControllerServiceProvider.validateEnableServices2:554 null TestStandardControllerServiceProvider.validateEnableServicesWithDisabledMissingService:614 null Tests in error: TestStandardProcessScheduler.testDisableControllerServiceWithProcessorTryingToStartUsingIt:207 » IllegalState TestStandardControllerServiceProvider.testConcurrencyWithEnablingReferencingServicesGraph:217->testEnableReferencingServicesGraph:261 » IllegalState TestStandardControllerServiceProvider.testEnableDisableWithReference:183 » IllegalState ``` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4295) When selecting a controller service for a processor, services that belong to the wrong scope are shown
[
https://issues.apache.org/jira/browse/NIFI-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129231#comment-16129231
]
ASF GitHub Bot commented on NIFI-4295:
--
Github user mcgilman commented on the issue:
https://github.com/apache/nifi/pull/2087
@markap14 It appears the changes introduced are causing some test failures.
```
Failed tests:
TestStandardControllerServiceProvider.validateEnableServices:506 null
TestStandardControllerServiceProvider.validateEnableServices2:554 null
TestStandardControllerServiceProvider.validateEnableServicesWithDisabledMissingService:614
null
Tests in error:
TestStandardProcessScheduler.testDisableControllerServiceWithProcessorTryingToStartUsingIt:207
» IllegalState
TestStandardControllerServiceProvider.testConcurrencyWithEnablingReferencingServicesGraph:217->testEnableReferencingServicesGraph:261
» IllegalState
TestStandardControllerServiceProvider.testEnableDisableWithReference:183
» IllegalState
```
> When selecting a controller service for a processor, services that belong to
> the wrong scope are shown
> --
>
> Key: NIFI-4295
> URL: https://issues.apache.org/jira/browse/NIFI-4295
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.4.0
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.4.0
>
>
> I have a Process Group, and in that Process Group I have defined a Controller
> Service. If I then move up to the root group and add a Processor, that takes
> a Controller Service of that type, I can now choose the Controller Service
> that is defined in a child group. When I do so, my processor becomes invalid
> and the reason given for the processor to be invalid is a NullPointer:
> {code}
> 2017-08-14 12:38:24,016 WARN [NiFi Web Server-76]
> o.a.n.controller.StandardProcessorNode Failed during validation
> java.lang.NullPointerException: null
> at
> org.apache.nifi.processor.StandardValidationContext.isValidationRequired(StandardValidationContext.java:143)
> at
> org.apache.nifi.components.PropertyDescriptor.validate(PropertyDescriptor.java:150)
> at
> org.apache.nifi.components.AbstractConfigurableComponent.validate(AbstractConfigurableComponent.java:103)
> at
> org.apache.nifi.controller.AbstractConfiguredComponent.validate(AbstractConfiguredComponent.java:329)
> at
> org.apache.nifi.controller.StandardProcessorNode.isValid(StandardProcessorNode.java:968)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatus(FlowController.java:2964)
> at
> org.apache.nifi.controller.FlowController.getGroupStatus(FlowController.java:2559)
> at
> org.apache.nifi.controller.FlowController.getGroupStatus(FlowController.java:2518)
> at
> org.apache.nifi.controller.FlowController.getGroupStatus(FlowController.java:2485)
> at
> org.apache.nifi.web.controller.ControllerFacade.getProcessGroupStatus(ControllerFacade.java:599)
> at
> org.apache.nifi.web.controller.ControllerFacade$$FastClassBySpringCGLIB$$5a42ba54.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
> at
> org.apache.nifi.web.controller.ControllerFacade$$EnhancerBySpringCGLIB$$344e3ba2.getProcessGroupStatus()
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.getProcessGroupFlow(StandardNiFiServiceFacade.java:3054)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
> at
> org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithReadLock(NiFiServiceFacadeLock.jav
[GitHub] nifi issue #2020: [NiFi-3973] Add PutKudu Processor for ingesting data to Ku...
Github user rickysaltzer commented on the issue: https://github.com/apache/nifi/pull/2020 I am going to be going through my final rounds of testing, so by the end of this week, I promise :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #125: MINIFI-368 exclude hidden files when scanning fo...
Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/125 reviewing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #2020: [NiFi-3973] Add PutKudu Processor for ingesting data to Ku...
Github user rickysaltzer commented on the issue: https://github.com/apache/nifi/pull/2020 @cammachusa - I updated my branch with a rebase, will you pull it down, make sure things are good for you and then force push it to this review? Thanks!! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #125: MINIFI-368 exclude hidden files when scan...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/125 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #125: MINIFI-368 exclude hidden files when scanning fo...
Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/125 looks good. replicated issue with vim editing. merged. thanks for fixing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #2020: [NiFi-3973] Add PutKudu Processor for ingesting data to Ku...
Github user cammachusa commented on the issue: https://github.com/apache/nifi/pull/2020 @rickysaltzer , I looked at your branch, it's identical with my branch. So sure, it will work. Just to clarify, what you mean by "pull it down, ... and force push it to this review"? Does that mean, pull your branch, and merge to mine, and push to Pull Request? Or pull your branch, and push to this PR, to clean up all previous commits to have just one? If the second, I haven't done cross push, can you help with what git commands I should run? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #2093: NIFI-4062 Provide an option to disable DTD validation for ...
Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/2093 @arunma Thanks for the updates. I'm going to build it locally once again just in case. When you rebase or squash the commits, then you need to push to your branch forcefully, for example: ``` # To rebase git checkout master git pull upstream master git checkout git rebase master # To squash (the HEAD~3 indicates how many recent commits you want to rebase) git rebase -i HEAD~3 # After rebase (with latest master, or squash commits), push updated commit to the existing PR with -f option git push -f origin ``` By pushing updates to the existing branch, you don't need to close/open new PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4062) Provide an option to disable DTD validation for EvaluateXPath and EvaluateXQuery
[ https://issues.apache.org/jira/browse/NIFI-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129566#comment-16129566 ] ASF GitHub Bot commented on NIFI-4062: -- Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/2093 @arunma Thanks for the updates. I'm going to build it locally once again just in case. When you rebase or squash the commits, then you need to push to your branch forcefully, for example: ``` # To rebase git checkout master git pull upstream master git checkout git rebase master # To squash (the HEAD~3 indicates how many recent commits you want to rebase) git rebase -i HEAD~3 # After rebase (with latest master, or squash commits), push updated commit to the existing PR with -f option git push -f origin ``` By pushing updates to the existing branch, you don't need to close/open new PR. > Provide an option to disable DTD validation for EvaluateXPath and > EvaluateXQuery > > > Key: NIFI-4062 > URL: https://issues.apache.org/jira/browse/NIFI-4062 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Arun Manivannan > Labels: beginner > > Currently EvaluateXPath and EvaluateXQuery can fail if the source document > has DOCTYPE definition in it, especially using external DTD. It would be > useful if these processor provides a property to disable DTD validation when > evaluate XPath or XQuery. > References: > https://community.hortonworks.com/questions/107121/does-evaluatexpath-support-doctype.html > https://stackoverflow.com/questions/243728/how-to-disable-dtd-at-runtime-in-javas-xpath -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2093: NIFI-4062 Provide an option to disable DTD validation for ...
Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/2093 @arunma Local contrib-check passed successfully. All review comments are addressed. LGTM, +1, merging to master. Thanks again for your contribution! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4062) Provide an option to disable DTD validation for EvaluateXPath and EvaluateXQuery
[ https://issues.apache.org/jira/browse/NIFI-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129581#comment-16129581 ] ASF GitHub Bot commented on NIFI-4062: -- Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/2093 @arunma Local contrib-check passed successfully. All review comments are addressed. LGTM, +1, merging to master. Thanks again for your contribution! > Provide an option to disable DTD validation for EvaluateXPath and > EvaluateXQuery > > > Key: NIFI-4062 > URL: https://issues.apache.org/jira/browse/NIFI-4062 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Arun Manivannan > Labels: beginner > > Currently EvaluateXPath and EvaluateXQuery can fail if the source document > has DOCTYPE definition in it, especially using external DTD. It would be > useful if these processor provides a property to disable DTD validation when > evaluate XPath or XQuery. > References: > https://community.hortonworks.com/questions/107121/does-evaluatexpath-support-doctype.html > https://stackoverflow.com/questions/243728/how-to-disable-dtd-at-runtime-in-javas-xpath -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2093: NIFI-4062 Provide an option to disable DTD validati...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2093 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4062) Provide an option to disable DTD validation for EvaluateXPath and EvaluateXQuery
[ https://issues.apache.org/jira/browse/NIFI-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129588#comment-16129588 ] ASF GitHub Bot commented on NIFI-4062: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2093 > Provide an option to disable DTD validation for EvaluateXPath and > EvaluateXQuery > > > Key: NIFI-4062 > URL: https://issues.apache.org/jira/browse/NIFI-4062 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Arun Manivannan > Labels: beginner > > Currently EvaluateXPath and EvaluateXQuery can fail if the source document > has DOCTYPE definition in it, especially using external DTD. It would be > useful if these processor provides a property to disable DTD validation when > evaluate XPath or XQuery. > References: > https://community.hortonworks.com/questions/107121/does-evaluatexpath-support-doctype.html > https://stackoverflow.com/questions/243728/how-to-disable-dtd-at-runtime-in-javas-xpath -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4062) Provide an option to disable DTD validation for EvaluateXPath and EvaluateXQuery
[ https://issues.apache.org/jira/browse/NIFI-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129587#comment-16129587 ] ASF subversion and git services commented on NIFI-4062: --- Commit 6df112e4b1c2495d281b1edfce84219e3bea581a in nifi's branch refs/heads/master from [~arunodhaya80] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=6df112e ] NIFI-4062 Provide an option to disable DTD validation for EvaluateXPath and EvaluateXQuery This closes #2093. Signed-off-by: Koji Kawamura > Provide an option to disable DTD validation for EvaluateXPath and > EvaluateXQuery > > > Key: NIFI-4062 > URL: https://issues.apache.org/jira/browse/NIFI-4062 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Arun Manivannan > Labels: beginner > > Currently EvaluateXPath and EvaluateXQuery can fail if the source document > has DOCTYPE definition in it, especially using external DTD. It would be > useful if these processor provides a property to disable DTD validation when > evaluate XPath or XQuery. > References: > https://community.hortonworks.com/questions/107121/does-evaluatexpath-support-doctype.html > https://stackoverflow.com/questions/243728/how-to-disable-dtd-at-runtime-in-javas-xpath -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4062) Provide an option to disable DTD validation for EvaluateXPath and EvaluateXQuery
[ https://issues.apache.org/jira/browse/NIFI-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-4062: Resolution: Fixed Fix Version/s: 1.4.0 Status: Resolved (was: Patch Available) > Provide an option to disable DTD validation for EvaluateXPath and > EvaluateXQuery > > > Key: NIFI-4062 > URL: https://issues.apache.org/jira/browse/NIFI-4062 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Arun Manivannan > Labels: beginner > Fix For: 1.4.0 > > > Currently EvaluateXPath and EvaluateXQuery can fail if the source document > has DOCTYPE definition in it, especially using external DTD. It would be > useful if these processor provides a property to disable DTD validation when > evaluate XPath or XQuery. > References: > https://community.hortonworks.com/questions/107121/does-evaluatexpath-support-doctype.html > https://stackoverflow.com/questions/243728/how-to-disable-dtd-at-runtime-in-javas-xpath -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #1968: NIFI-3931 - Added EL to properties in SFTP transfer
Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/1968 Thanks for the update @pvillard31 , LGTM +1, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3931) putSftp process port property should support for expression language
[ https://issues.apache.org/jira/browse/NIFI-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129597#comment-16129597 ] ASF GitHub Bot commented on NIFI-3931: -- Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/1968 Thanks for the update @pvillard31 , LGTM +1, merging to master! > putSftp process port property should support for expression language > > > Key: NIFI-3931 > URL: https://issues.apache.org/jira/browse/NIFI-3931 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.2.0 >Reporter: Cheng Chin Tat >Assignee: Pierre Villard >Priority: Minor > Labels: easyfix > Attachments: TestFTP.xml > > > PutSftp Processor port property should support for expression language so > that dynamic port number can be pass to the processor during run time. > Rather than preset the port on design time. > This changes involve changing the PropertyDescriptor SFTP_PORT validator > StandardValidators.NON_NEGATIVE_INTEGER_VALIDATOR to > StandardValidators.NON_EMPTY_VALIDATOR and other codes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #1968: NIFI-3931 - Added EL to properties in SFTP transfer
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1968 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3931) putSftp process port property should support for expression language
[ https://issues.apache.org/jira/browse/NIFI-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129603#comment-16129603 ] ASF subversion and git services commented on NIFI-3931: --- Commit 9f23ec360acd2bf4d949f08b3eb0cea3decc5263 in nifi's branch refs/heads/master from [~pvillard] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9f23ec3 ] NIFI-3931 - Added EL to properties in SFTP transfer The properties are already evaluated against EL. Also added an integration test that is failing without the change. This closes #1968. Signed-off-by: Koji Kawamura > putSftp process port property should support for expression language > > > Key: NIFI-3931 > URL: https://issues.apache.org/jira/browse/NIFI-3931 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.2.0 >Reporter: Cheng Chin Tat >Assignee: Pierre Villard >Priority: Minor > Labels: easyfix > Attachments: TestFTP.xml > > > PutSftp Processor port property should support for expression language so > that dynamic port number can be pass to the processor during run time. > Rather than preset the port on design time. > This changes involve changing the PropertyDescriptor SFTP_PORT validator > StandardValidators.NON_NEGATIVE_INTEGER_VALIDATOR to > StandardValidators.NON_EMPTY_VALIDATOR and other codes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-3931) putSftp process port property should support for expression language
[ https://issues.apache.org/jira/browse/NIFI-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129604#comment-16129604 ] ASF GitHub Bot commented on NIFI-3931: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1968 > putSftp process port property should support for expression language > > > Key: NIFI-3931 > URL: https://issues.apache.org/jira/browse/NIFI-3931 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.2.0 >Reporter: Cheng Chin Tat >Assignee: Pierre Villard >Priority: Minor > Labels: easyfix > Fix For: 1.4.0 > > Attachments: TestFTP.xml > > > PutSftp Processor port property should support for expression language so > that dynamic port number can be pass to the processor during run time. > Rather than preset the port on design time. > This changes involve changing the PropertyDescriptor SFTP_PORT validator > StandardValidators.NON_NEGATIVE_INTEGER_VALIDATOR to > StandardValidators.NON_EMPTY_VALIDATOR and other codes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-3931) putSftp process port property should support for expression language
[ https://issues.apache.org/jira/browse/NIFI-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-3931: Resolution: Fixed Fix Version/s: 1.4.0 Status: Resolved (was: Patch Available) > putSftp process port property should support for expression language > > > Key: NIFI-3931 > URL: https://issues.apache.org/jira/browse/NIFI-3931 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.2.0 >Reporter: Cheng Chin Tat >Assignee: Pierre Villard >Priority: Minor > Labels: easyfix > Fix For: 1.4.0 > > Attachments: TestFTP.xml > > > PutSftp Processor port property should support for expression language so > that dynamic port number can be pass to the processor during run time. > Rather than preset the port on design time. > This changes involve changing the PropertyDescriptor SFTP_PORT validator > StandardValidators.NON_NEGATIVE_INTEGER_VALIDATOR to > StandardValidators.NON_EMPTY_VALIDATOR and other codes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2055: NIFI-4028 - fix cache update when Wait releases flow files
Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/2055 @pvillard31 Local contrib check passed without issue, and tested with different flows. LGTM +1, merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4028) Wait processor removes cache regardless of the remaing counter
[
https://issues.apache.org/jira/browse/NIFI-4028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129665#comment-16129665
]
ASF GitHub Bot commented on NIFI-4028:
--
Github user ijokarumawak commented on the issue:
https://github.com/apache/nifi/pull/2055
@pvillard31 Local contrib check passed without issue, and tested with
different flows. LGTM +1, merging to master. Thanks!
> Wait processor removes cache regardless of the remaing counter
> --
>
> Key: NIFI-4028
> URL: https://issues.apache.org/jira/browse/NIFI-4028
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Koji Kawamura
>Assignee: Pierre Villard
> Attachments: WaitNotify_NIFI-4028.xml
>
>
> Current [Wait processor implementation removes cache
> entry|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/Wait.java#L438]
> if it's configured with 'Releasable FlowFile Count' is one, when it finds a
> wait signal reaches to 'Target Signal Count' no matter how large the current
> count is.
> This behavior can be problematic, when an user expects the same Signal
> Identifier to be released multiple FlowFiles one by one while multiple counts
> to be accumulated by Notify.
> For example this scenario does not work:
> {code}
> 1. Notify notifies to a 'signal-a' + 1 => 'signal-a' = 1
> 2. Notify notifies to a 'signal-a' again, +1 => 'signal-a' = 2
> 3. Wait releases 1 FlowFile, then remove the 'signal-a' => 'signal-a' is
> removed
> 4. Wait will not be able to release another FlowFile
> {code}
> This should be fixed as follows:
> {code}
> 1. Notify notifies to a 'signal-a' + 1 => 'signal-a' = 1
> 2. Notify notifies to a 'signal-a' again, +1 => 'signal-a' = 2
> 3. Wait releases 1 FlowFile, then update the 'signal-a', -1 => 'signal-a' = 1
> 4. Wait releases another FlowFile, then 'signal-a' becomes 0, remove
> 'signal-a'
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] nifi pull request #2055: NIFI-4028 - fix cache update when Wait releases flo...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2055 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4028) Wait processor removes cache regardless of the remaing counter
[
https://issues.apache.org/jira/browse/NIFI-4028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129666#comment-16129666
]
ASF subversion and git services commented on NIFI-4028:
---
Commit c1b99d584d93bb6f26ba0986f2fcaf663b0caef3 in nifi's branch
refs/heads/master from [~pvillard]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=c1b99d5 ]
NIFI-4028 - fix cache update when Wait releases flow files
NIFI-4028: Refactored Wait processor.
- Consolidated implementation for the cases of releasableFlowCount is 1 or
more, in order to reduce complexity and behavior differences
- Added 'consumed' counter when total counter is used to release incoming
FlowFiles
- Fixed method name typo, releaseCandidates
This closes #2055.
Signed-off-by: Koji Kawamura
> Wait processor removes cache regardless of the remaing counter
> --
>
> Key: NIFI-4028
> URL: https://issues.apache.org/jira/browse/NIFI-4028
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Koji Kawamura
>Assignee: Pierre Villard
> Attachments: WaitNotify_NIFI-4028.xml
>
>
> Current [Wait processor implementation removes cache
> entry|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/Wait.java#L438]
> if it's configured with 'Releasable FlowFile Count' is one, when it finds a
> wait signal reaches to 'Target Signal Count' no matter how large the current
> count is.
> This behavior can be problematic, when an user expects the same Signal
> Identifier to be released multiple FlowFiles one by one while multiple counts
> to be accumulated by Notify.
> For example this scenario does not work:
> {code}
> 1. Notify notifies to a 'signal-a' + 1 => 'signal-a' = 1
> 2. Notify notifies to a 'signal-a' again, +1 => 'signal-a' = 2
> 3. Wait releases 1 FlowFile, then remove the 'signal-a' => 'signal-a' is
> removed
> 4. Wait will not be able to release another FlowFile
> {code}
> This should be fixed as follows:
> {code}
> 1. Notify notifies to a 'signal-a' + 1 => 'signal-a' = 1
> 2. Notify notifies to a 'signal-a' again, +1 => 'signal-a' = 2
> 3. Wait releases 1 FlowFile, then update the 'signal-a', -1 => 'signal-a' = 1
> 4. Wait releases another FlowFile, then 'signal-a' becomes 0, remove
> 'signal-a'
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4028) Wait processor removes cache regardless of the remaing counter
[
https://issues.apache.org/jira/browse/NIFI-4028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129667#comment-16129667
]
ASF subversion and git services commented on NIFI-4028:
---
Commit c1b99d584d93bb6f26ba0986f2fcaf663b0caef3 in nifi's branch
refs/heads/master from [~pvillard]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=c1b99d5 ]
NIFI-4028 - fix cache update when Wait releases flow files
NIFI-4028: Refactored Wait processor.
- Consolidated implementation for the cases of releasableFlowCount is 1 or
more, in order to reduce complexity and behavior differences
- Added 'consumed' counter when total counter is used to release incoming
FlowFiles
- Fixed method name typo, releaseCandidates
This closes #2055.
Signed-off-by: Koji Kawamura
> Wait processor removes cache regardless of the remaing counter
> --
>
> Key: NIFI-4028
> URL: https://issues.apache.org/jira/browse/NIFI-4028
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Koji Kawamura
>Assignee: Pierre Villard
> Attachments: WaitNotify_NIFI-4028.xml
>
>
> Current [Wait processor implementation removes cache
> entry|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/Wait.java#L438]
> if it's configured with 'Releasable FlowFile Count' is one, when it finds a
> wait signal reaches to 'Target Signal Count' no matter how large the current
> count is.
> This behavior can be problematic, when an user expects the same Signal
> Identifier to be released multiple FlowFiles one by one while multiple counts
> to be accumulated by Notify.
> For example this scenario does not work:
> {code}
> 1. Notify notifies to a 'signal-a' + 1 => 'signal-a' = 1
> 2. Notify notifies to a 'signal-a' again, +1 => 'signal-a' = 2
> 3. Wait releases 1 FlowFile, then remove the 'signal-a' => 'signal-a' is
> removed
> 4. Wait will not be able to release another FlowFile
> {code}
> This should be fixed as follows:
> {code}
> 1. Notify notifies to a 'signal-a' + 1 => 'signal-a' = 1
> 2. Notify notifies to a 'signal-a' again, +1 => 'signal-a' = 2
> 3. Wait releases 1 FlowFile, then update the 'signal-a', -1 => 'signal-a' = 1
> 4. Wait releases another FlowFile, then 'signal-a' becomes 0, remove
> 'signal-a'
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4028) Wait processor removes cache regardless of the remaing counter
[
https://issues.apache.org/jira/browse/NIFI-4028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129669#comment-16129669
]
ASF GitHub Bot commented on NIFI-4028:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/2055
> Wait processor removes cache regardless of the remaing counter
> --
>
> Key: NIFI-4028
> URL: https://issues.apache.org/jira/browse/NIFI-4028
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Koji Kawamura
>Assignee: Pierre Villard
> Attachments: WaitNotify_NIFI-4028.xml
>
>
> Current [Wait processor implementation removes cache
> entry|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/Wait.java#L438]
> if it's configured with 'Releasable FlowFile Count' is one, when it finds a
> wait signal reaches to 'Target Signal Count' no matter how large the current
> count is.
> This behavior can be problematic, when an user expects the same Signal
> Identifier to be released multiple FlowFiles one by one while multiple counts
> to be accumulated by Notify.
> For example this scenario does not work:
> {code}
> 1. Notify notifies to a 'signal-a' + 1 => 'signal-a' = 1
> 2. Notify notifies to a 'signal-a' again, +1 => 'signal-a' = 2
> 3. Wait releases 1 FlowFile, then remove the 'signal-a' => 'signal-a' is
> removed
> 4. Wait will not be able to release another FlowFile
> {code}
> This should be fixed as follows:
> {code}
> 1. Notify notifies to a 'signal-a' + 1 => 'signal-a' = 1
> 2. Notify notifies to a 'signal-a' again, +1 => 'signal-a' = 2
> 3. Wait releases 1 FlowFile, then update the 'signal-a', -1 => 'signal-a' = 1
> 4. Wait releases another FlowFile, then 'signal-a' becomes 0, remove
> 'signal-a'
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (NIFI-4028) Wait processor removes cache regardless of the remaing counter
[
https://issues.apache.org/jira/browse/NIFI-4028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Koji Kawamura updated NIFI-4028:
Resolution: Fixed
Fix Version/s: 1.4.0
Status: Resolved (was: Patch Available)
> Wait processor removes cache regardless of the remaing counter
> --
>
> Key: NIFI-4028
> URL: https://issues.apache.org/jira/browse/NIFI-4028
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Koji Kawamura
>Assignee: Pierre Villard
> Fix For: 1.4.0
>
> Attachments: WaitNotify_NIFI-4028.xml
>
>
> Current [Wait processor implementation removes cache
> entry|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/Wait.java#L438]
> if it's configured with 'Releasable FlowFile Count' is one, when it finds a
> wait signal reaches to 'Target Signal Count' no matter how large the current
> count is.
> This behavior can be problematic, when an user expects the same Signal
> Identifier to be released multiple FlowFiles one by one while multiple counts
> to be accumulated by Notify.
> For example this scenario does not work:
> {code}
> 1. Notify notifies to a 'signal-a' + 1 => 'signal-a' = 1
> 2. Notify notifies to a 'signal-a' again, +1 => 'signal-a' = 2
> 3. Wait releases 1 FlowFile, then remove the 'signal-a' => 'signal-a' is
> removed
> 4. Wait will not be able to release another FlowFile
> {code}
> This should be fixed as follows:
> {code}
> 1. Notify notifies to a 'signal-a' + 1 => 'signal-a' = 1
> 2. Notify notifies to a 'signal-a' again, +1 => 'signal-a' = 2
> 3. Wait releases 1 FlowFile, then update the 'signal-a', -1 => 'signal-a' = 1
> 4. Wait releases another FlowFile, then 'signal-a' becomes 0, remove
> 'signal-a'
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-3484) GenerateTableFetch Should Allow for Right Boundary
[ https://issues.apache.org/jira/browse/NIFI-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129707#comment-16129707 ] ASF GitHub Bot commented on NIFI-3484: -- Github user patricker commented on the issue: https://github.com/apache/nifi/pull/2091 @pvillard31 This looks good. > GenerateTableFetch Should Allow for Right Boundary > -- > > Key: NIFI-3484 > URL: https://issues.apache.org/jira/browse/NIFI-3484 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > > When using GenerateTableFetch it places no right hand boundary on pages of > data. This can lead to issues when the statement says to get the next 1000 > records greater then a specific key, but records were added to the table > between the time the processor executed and when the SQL is being executed. > As a result it pulls in records that did not exist when the processor was > run. On the next execution of the processor these records will be pulled in > a second time. > Example: > Partition Size = 1000 > First run (no state): Count(*)=4700 and MAX(ID)=4700. > 5 FlowFiles are generated, the last one will say to fetch 1000, not 700. (But > I don't think this is really a bug, just an observation). > 5 Flow Files are now in queue to be executed by ExecuteSQL. Before the 5th > file can execute 400 new rows are added to the table. When the final SQL > statement is executed 300 extra records, with higher ID values, will also be > pulled into NiFi. > Second run (state: ID=4700). Count(*) ID>4700 = 400 and MAX(ID)=5100. > 1 Flow File is generated, but includes 300 records already pulled into NiFI. > The solution is to have an optional property that will let users use the new > MAX(ID) as a right boundary when generating queries. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2091: NIFI-3484 GenerateTableFetch Should Allow for Right Bounda...
Github user patricker commented on the issue: https://github.com/apache/nifi/pull/2091 @pvillard31 This looks good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #2093: NIFI-4062 Provide an option to disable DTD validation for ...
Github user arunma commented on the issue: https://github.com/apache/nifi/pull/2093 Thanks a ton for the merge and the git tip, @ijokarumawak!! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4062) Provide an option to disable DTD validation for EvaluateXPath and EvaluateXQuery
[ https://issues.apache.org/jira/browse/NIFI-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129726#comment-16129726 ] ASF GitHub Bot commented on NIFI-4062: -- Github user arunma commented on the issue: https://github.com/apache/nifi/pull/2093 Thanks a ton for the merge and the git tip, @ijokarumawak!! > Provide an option to disable DTD validation for EvaluateXPath and > EvaluateXQuery > > > Key: NIFI-4062 > URL: https://issues.apache.org/jira/browse/NIFI-4062 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Arun Manivannan > Labels: beginner > Fix For: 1.4.0 > > > Currently EvaluateXPath and EvaluateXQuery can fail if the source document > has DOCTYPE definition in it, especially using external DTD. It would be > useful if these processor provides a property to disable DTD validation when > evaluate XPath or XQuery. > References: > https://community.hortonworks.com/questions/107121/does-evaluatexpath-support-doctype.html > https://stackoverflow.com/questions/243728/how-to-disable-dtd-at-runtime-in-javas-xpath -- This message was sent by Atlassian JIRA (v6.4.14#64029)
